[Development Log] Making Agents Debate Each Other — Roundtable Design and Limitations

119.216.***.***
10

This is tunaFlow technical post series #4. In part 3, we covered Branch structure, and Roundtable (RT) is an extension mode of Branch. When branches.mode = "roundtable", multiple agents discuss a single topic.


Why Have Agents Discuss With Each Other

When you ask an agent to review code, it mostly says "looks good." If you entrust design to a single agent, it's structurally difficult for them to self-critique their own proposals.

Code review works in human teams because the reviewer is a different person from the author. They have different perspectives and miss different things. We can apply the same structure to agents — when multiple agents with different engines, different roles, and different prompts look at the same topic, what one agent misses, another catches.

RT is the implementation of this idea.


RT = Extension Mode of Branch

RT is not a separate system but a mode of Branch.

// branches table
pub struct Branch {
    pub mode: String,  // "chat" or "roundtable"
    // ... rest is identical to regular Branch
}

When you create an RT, a Branch with mode = "roundtable" is created, and participants' messages accumulate in the shadow conversation. In the sidebar it appears in the Roundtables section, opens in the drawer, and adopt works identically. It uses the Branch infrastructure covered in part 3.

The difference is messages are not sent by the user but by participants sequentially (or simultaneously).


Two Discussion Modes

Sequential — Sequential Discussion

Participants speak one by one in order. Later participants can see what earlier participants said.

Round 1:
  Architect (claude)  → Design proposal: "Separate into 3 modules"
  Reviewer (codex)    → Seeing Architect's response: "Module boundaries are ambiguous"
  Verifier (gemini)   → Seeing both: "Agree with Reviewer, module 2 is problematic"
// sequential.rs — core loop
for p in participants {
    let prompt = if p.blind {
        // blind participant: doesn't see other participants' responses
        build_round_prompt_with_identity(topic, &[], &[], Some(&identity))
    } else {
        // includes responses from earlier participants
        build_round_prompt_with_identity(topic, transcript, &round_responses, Some(&identity))
    };
    let result = stream_participant(p, prompt, ...).await;
    round_responses.push((p.name.clone(), result.content.clone()));
}

The advantage of sequential mode is that the conversation builds naturally. Later participants can refute or supplement earlier points.

Deliberative — Parallel Discussion

Participants speak simultaneously. They cannot see each other's responses.

Round 1:
  Architect (claude)  → (independently) "Separate into 3 modules"
  Reviewer (codex)    → (independently) "Interface abstraction needed"
  Verifier (gemini)   → (independently) "Insufficient test coverage"
// deliberative.rs — parallel execution with tokio::spawn
for p in participants {
    let prompt = build_round_prompt_with_identity(topic, transcript, &[], Some(&identity));
    //                                                                 ^^^ no current round responses
    tokio::spawn(async move {
        let result = stream_participant(p, prompt, ...).await;
        tx.send((msg_id, result)).await;
    });
}
// collect results in completion order
while let Some((id, result)) = rx.recv().await { ... }

The advantage of parallel mode is independent perspectives. Since earlier participants' opinions don't influence them, you get truly different viewpoints. The downside is that points of discussion can scatter.

When to Use Which Mode

Situation

Recommended Mode

Reason

Code review

Sequential

Reviewer must see and evaluate Developer results

Design brainstorming

Deliberative

Independent ideas needed

Security audit

Deliberative + blind

Independent evaluation without preconceptions

Consensus building

Sequential 2 rounds

Round 1: independent opinions, Round 2: mutual reactions


Participant Execution Model

Each participant runs as an independent subprocess. Not PTY session sharing but separate -p mode invocations per participant.

RT execution:
  Participant A (claude) → claude -p "..." → subprocess 1
  Participant B (codex)  → codex exec "..." → subprocess 2
  Participant C (gemini) → gemini -p "..." → subprocess 3

Reason for this design: RT participants can be different engines. Mixing Claude, Codex, and Gemini for discussion is the core value of RT. A PTY session is one per engine, so sharing is structurally impossible.

Execution handlers per engine:

Engine

Execution Method

Streaming

Claude

claude::stream_run()

Real-time chunks

Codex

codex::stream_run()

Real-time chunks

Gemini

gemini::stream_run()

Real-time chunks

Ollama

openai_compat::stream_run()

Async native

OpenCode

opencode::run()

Non-streaming

During execution, roundtable:chunk events stream to the frontend, displaying each participant's response in real-time in the drawer.


Context Management: Minimum Context Strategy

Passing the entire ContextPack (~5-7k tokens) to RT participants is wasteful. With 3 participants it's 3x, with 2 rounds it's 6x. So RT uses a minimum context strategy.

RtContextCache — Tier 0+1 Only

// context.rs
pub struct RtContextCache {
    pub context: Option,  // ~100-200 tokens
}

Of the 12 sections in ContextPack, only project path + active Plan summary are included. Skills, retrieval, compressed memory, cross-session and more are all excluded. For RT participants, just knowing "what project and what task are we working on now" is enough.

Vector Search to Compress Previous Rounds

Passing complete transcripts from round 2+ onwards causes token explosion. If 3 participants each respond with 4,000 characters, that's 12,000 characters (~3,000 tokens).

RtVectorIndex embeds previous round responses and extracts only the top 5 chunks related to the current topic.

// context.rs
pub fn search(&self, topic: &str, limit: usize) -> Vec<(String, String)> {
    // cosine similarity search, only >= 0.2
    // truncate each chunk to 800 characters
}

Result: 12,000 characters → 2,400 characters. ~80% token savings. Falls back to full transcript if the rawq daemon is down.


Role System

Assigning roles to participants injects role-specific guidelines into the prompt and limits output tokens.

Four Roles

fn role_guidance(role: &str) -> &'static str {
    match role {
        "proposer" => "State conclusions first, then provide reasoning...",
        "reviewer" | "critic" => "Evaluate across 4 dimensions (plan_coverage, code_quality, test_coverage, convention)...",
        "verifier" | "judge" => "Judge independently without relying on other participants' evaluations...",
        "synthesizer" | "lead" => "Organize into 3 sections: consensus/contested/dissent...",
        _ => "",
    }
}

Output Limits per Role

Role

Default Token Limit

Reason

proposer

1,200

Core proposal only

reviewer/critic

900

Focus on evaluation

verifier/judge

800

Verdicts should be brief

synthesizer/lead

2,000

Synthesis can be longer

Token limits are injected at the prompt start as [Output limit: Keep your response under approximately N tokens.]. It's a directive, not enforcement, but most models follow it well.

Blind Participants

When set to blind: true, that participant receives only the topic and their own role and cannot see other participants' responses at all.

if p.blind {
    build_round_prompt_with_identity(topic, &[], &[], Some(&identity))
    //                                       ^^   ^^ both transcript and current round are empty arrays
}

Useful for security audits or independent validation. It structurally blocks the bias of "another reviewer approved it so I will too."


Review RT: Usage in Workflow

In the workflow pipeline covered in part 2, the Review stage uses RT.

Detect impl-complete
  → Create Review Branch (mode: "roundtable")
  → Execute Reviewer + Verifier (2 participants)
  → Derive verdict (pass/fail/conditional)

Review RT has 2 participants because the roles are distinctly different. The Reviewer evaluates code quality/tests/conventions numerically, and the Verifier independently validates the Reviewer's evaluation itself. 3+ participants only raise token costs without adding substantial new perspectives. Unlike design discussions, review has fixed criteria, so accuracy matters more than perspective diversity.


Brief: Summarizing Discussion Results

When RT ends, a rule-based summary (brief) is auto-generated. No additional LLM calls.

// persist.rs — save_shared_brief
for name in &unique_names {
    if let Some((_, content)) = transcript.iter().rev().find(|(n, _)| n == name) {
        let summary = first_sentences(content, 2);  // first 2 sentences, 300 char limit
        position_lines.push(format!("- **{}**: {}", name, summary));
    }
}

Extract the first 2 sentences from each participant's last response and organize as Key Positions. The brief is saved in the memos table as type = 'roundtable_brief', and these Key Positions are inserted into the main conversation during adopt.

Why not use LLM summarization: RT itself already consumes significant tokens (~10k-20k per 3 participants, 1 round). Adding a summary call increases costs further. First 2 sentences is crude, but since RT participants follow the role guideline "state conclusions first," the first 2 sentences actually contain their core position.


Actual Usage Patterns

Round Operation

A common pattern in 3-person discussions is minimum 5 rounds. It's not just agents discussing — users intervene mid-discussion to clarify direction. User intervention counts as a round too.

Round 1: 3 agents each present opinions
Round 2: User — "Direction A seems promising, but let's reconsider B's concerns and review"
Round 3: 3 agents — review counterarguments + modify/supplement own opinions
Round 4: User — "This part seems agreed, let's focus on remaining disputes"
Round 5: 3 agents — final opinions + synthesizer summary

With 2 user interventions, actual agent discussion is 3 rounds. That's usually enough for convergence. Participants agree with each other, modify their proposals, supplement others' proposals. Cases where they just repeat "I maintain my opinion" are rarer than expected.

Complex design discussions may run 8-10 rounds with users clarifying direction multiple times. The synthesizer summarizes what was decided and what wasn't at the end. If consensus fails on certain issues, the user makes the call.

For simpler topics or when direction is already set, 1:1 conversation with Architect is sufficient — no need for RT.

Real Examples

During tunaFlow development, RT proved effective designing the LLM Wiki implementation for tunapi (API project) and seCall (call analysis project). Three agents proposed different approaches, and over rounds reviewed and supplemented each other's proposals, with results directly implemented. Had we discussed just with Architect alone, we might have pursued one direction deeply and missed other possibilities that were actually compared and evaluated through discussion.

User's Role

In RT, the user is a moderator. Rather than expecting agents to auto-reach consensus, interjecting between rounds to set direction and clarify disputes raises discussion quality. One-liners like "let's narrow to this direction" or "merge A's proposal with B's concern" significantly improve the next round's quality.


Current Limitations

1. Token Cost Proportional to Rounds

N participants × R rounds generates N×R LLM calls. 3 participants, 3 rounds = 9 calls. Since previous round transcripts are included, input tokens grow as rounds progress. Vector search cuts prior round context ~80%, but fundamentally the cost structure scales with N and R.

That said, the RT discussion cost is reasonable compared to fixing wrong designs later. The problem is hard to predict cost upfront. Whether it takes 3 or 8 rounds depends on topic complexity.

2. Sequential Mode Visual Limitation

In sequential discussion, when participant B responds, they're clearly reacting to participant A. They quote, refute, or supplement earlier points. But the UI shows "responses appearing in order," so the actual interaction between participants isn't visually obvious.

Participant B's prompt includes A's full response, and B digests it before responding, but this "digestion" happens inside the LLM. The output shows good cross-referencing, but real-time viewing may feel like independent responses.


Core Decisions Summary

Decision

Reason

RT = Branch mode

No separate system needed. Reuse Branch infrastructure (shadow conv, drawer, adopt)

Independent subprocess per participant

Allow mixing different engines. PTY session sharing structurally impossible

Minimal context (RtContextCache)

Context cost grows with participants × rounds. Use Tier 0+1 only

Vector search to compress transcript

Passing full prior round explodes tokens. ~80% reduction

Output limit per role

Prevent verbose responses. proposer 1200, reviewer 900, synthesizer 2000

Rule-based brief (no LLM)

RT already consumes many tokens. Avoid extra calls

Review RT with 2 participants

Review prioritizes accuracy over perspective diversity. Reviewer + independent Verifier


Next Edition Preview

Part 5: "When Conversations Get Long" — Implementing Long-Term Memory for Agents

Longer agent conversations overflow the context window. tunaFlow implements long-term memory via compressed memory, vector search, and cross-session links. We'll cover compression timing, search quality, and the design question "how much history should an agent remember?"


Series Index

  1. [Give Agents Process] - What We Learned Building AOC

  2. [Plan → Dev → Review] — Implementing Workflow Pipeline

  3. Branching Conversations — Branch Design and Usage

  4. Making Agents Discuss — Roundtable Design and Limitations (this post)

  5. When Conversations Get Long — Implementing Agent Long-Term Memory

  6. Running Workflow on $20 Claude — Engine Architecture

  7. Teaching Agents Code Structure — rawq + code-review-graph

  8. Applying Only Needed Skills from 246 — Auto-Selecting Skills Implementation

  9. When Agents Repeat Mistakes — Quality Assurance Design

  10. Full Cycle Run with tunaFlow — Workflow Live Test Retrospective

  • [Published] ContextPack — Context Design for Multi-Agent Orchestrator


References

  • tunaFlow Internal Documentation

    • RT Execution Engine: src-tauri/src/commands/roundtable_helpers/executor.rs

    • Sequential Mode: src-tauri/src/commands/roundtable_helpers/sequential.rs

    • Deliberative Mode: src-tauri/src/commands/roundtable_helpers/deliberative.rs

    • Context Cache: src-tauri/src/commands/roundtable_helpers/context.rs

    • Role Guidance: src-tauri/src/commands/roundtable_helpers/types.rs

    • Prompt Assembly: src-tauri/src/commands/roundtable_helpers/prompt.rs

    • Brief Generation: src-tauri/src/commands/roundtable_helpers/persist.rs

로그인한 회원만 댓글 등록이 가능합니다.

개발한당

KR | ID | EN
  • IDR
  • KOR
8.34 0.01

2026.07.10 KEB 하나은행 고시회차 1330회

다가오는 한인 행사일정

  • 등록 된 일정이 없어요!