[Bonus Episode] I Had Gemini Do a Code Review and Here's What Happened After 22 Minutes of Deep Thinking

112.63.***.***
9

This is a side story from the tunaFlow development series. Since this isn't a regular installment but rather a breather post, I'm going to write in great detail this time. Please be aware of the long length (though it's entertaining haha). If anyone has experienced similar situations, you might relate. If you rebut, your points are all correct too. 😁

First, the "rebuttals" and "verifications" that appear in this post are not strictly speaking something I did directly myself. What I did was **simply copy-paste the conclusions** from exchanges between external Gemini and Claude (who plays the coder role) in my project. (I've been feeling myself turning into more of a copy-paste machine ever since paying for AI from ChatGPT's early days.) I was so reluctant to even do that, which is why I eventually created tunaFlow, but today I ran it manually. During that process, there were so many funny moments that I'm recording them as a memoir.


The Setup: A Disaster Caused by One Word

Before public beta or publishing to a repo, I always have external agents do a review pass. (I do rolling reviews in practice too. If you batch them later, technical debt accumulates) In the internal Roundtable, no matter how many times we go back and forth, models from the same family are too polite with each other. When I show Claude code I've written, it starts with "Overall, this is well designed." (Really)

So I frequently used the phrase asking Gemini to point out "problems only, harshly." I also added that it would be good from a senior engineer's perspective. These two words were the source of all incidents today.

To get straight to the point, the combination of "harshly + senior" was a combo key that unlocked the LLM's assertion safety rail. I discovered this for the first time today too.


Round 1: Five Shots Fired

Gemini's first response was a cleanly organized 400-character review. It contained 5 points of criticism.

1. Using HTML comment markers to control workflow is the worst antipattern

The moment the LLM makes a typo in the marker (`-` etc.) or forgets a closing tag, the entire orchestration pipeline collapses. It's a very fragile structure.

2. Fake test coverage

The entire syntax coverage is only 20.5%. (...) Essentially 'shell test' that only runs pure functions within the frontend.

3. Orchestration state is dependent on the frontend

When you refresh the UI window, the orchestration loop evaporates.

4. Fatal lack of sandbox

I checked because the file `src-tauri/src/guardrail.rs` existed, but it's just a string processing logic that cuts the maximum character count of the prompt context. There's no isolation layer to protect the host OS when trying to execute destructive commands like `rm -rf`.

5. The illusion of Reviewer cross-validation

If Developer hallucinates and lies saying "test passed," the Reviewer, who doesn't even have permission to execute code, just reads that text and issues a pass judgment as-is.

The final overall assessment was like this. 😇

| "Absolutely cannot go to public beta."

I was quite shocked upon receiving it. Of the 5 points, one or two aren't wrong. Especially #5 is correct. I knew it too, and it was already decided to work on it after RT optimization.


"The problem was the rest."


Rebuttal Verification: Made Claude Expose the Code

At this point, what I did was copy all 5 of Gemini's points as-is and pass them to Opus, who is handling the coder role in tunaFlow, asking him to "verify each one with code." I just received the results in the middle.

But the verification results Claude gave back were interesting. Claude tends to become somewhat passive when receiving external reviews, but when given the condition to "rebut with code as evidence," he became a completely different person. 🫢 It's no secret that I opened code while making tunaFlow for the first time. (It's a test project that started from the principle of never looking at code normally and never modifying it either. That's why the sidebar explorer shows DOCS rather than FILES)

#1 HTML Marker — Theoretically Vulnerable, Practically Stable

The marker method is true. `src/lib/planProposalParser.ts` parses comments like `` using regex. But what Claude pointed out was something Gemini missed.

The CLI subprocess environment has no function calling protocol. The stdout of Claude/Codex/Gemini CLI just has plain text. To receive structured tool-use, you need to go directly through the SDK, but tunaFlow's philosophy is to use the user's \$20 subscription as-is. SDK direct = API charges (super expensive).

In other words, markers were a rational choice under CLI-first constraints, not an "antipattern." And I've never had a pipeline collapse due to marker typos throughout development. There might have been one in the early PoC, but after prompt tuning, there's never been a single "actual failure."

#2 Test Coverage — Half Right

The 20.5% FE coverage is correct. Instructing `invoke` to be handled as `vi.mock` is also correct. But there are 259 unit tests separately in the Rust backend. ContextPack assembly, DB migration, JSONL parsing, vector search, etc. — core logic is verified in Rust.

Gemini only ran `npm run test:coverage` and concluded it was "fake test." They didn't run `cargo test --lib`. Yet `src-tauri/tests/` is plainly visible in the file tree.

#3 Frontend Dependency — This One's True

The event listener in `runtimeSlice.ts` is attached to Zustand, so `runningThreadIds` always initializes to `[]` on app restart. This is true, and I've already written documentation up to P1 in `docs/plans/engineServerModeArchitecturePlan.md`. However, thanks to the DB = SSOT principle, the plan/phase/event themselves survive and can resume. "Evaporates" is an exaggeration, but "needs improvement" is accurate.

#4 Sandbox Absence — Gemini Stumbles Decisively Here

Gemini saw the file `src-tauri/src/guardrail.rs` and guessed "this must be the sandbox." Then they opened the file and seemed to conclude "oh, character limit. So there's no sandbox after all." (So I checked it too)

Except for the filename being `guardrail.rs`, it has nothing to do with a sandbox, yet **Gemini deduced functionality from the filename.** That file is just a utility that trims prompt size to prevent ContextPack from getting too large.

The real sandbox is provided by the CLI engine itself.

- Claude: `--permission-mode ask` + approval UI

- Codex: `--full-auto` / `--sandbox` flags (OpenAI's own sandbox rules)

tunaFlow calls CLI as a subprocess, so the CLI's approval layer works as-is. OpenHands/Devin are **SaaS cloud-based** forms, so container isolation is essential. tunaFlow is a local desktop AOC(and I gave it that grandiose name) so the positioning is different. It's just orchestrating an "environment where users are already directly using Claude CLI on their own PC," and adding containers to that is overengineering.

"Gemini completely ignored this positioning difference."

#5 Reviewer Illusion — This One's Really Right

This was a rebuttal-proof point. It actually says this in `src/lib/workflow/reviewWorkflow.ts:58`.

`You are a code reviewer. **Read the code** and verify it. Do not directly execute build/test commands.`,

And on line 71:

2. Verify Verification Results: Confirm the verification results reported by Developer.

That is, if Developer hallucinates and says "test passed," Reviewer trusts that text and issues pass. The README says "overcome self-validation limitations through 2-agent cross-validation," but the actual implementation is at the level of "a person who reads their own report."

If I make an excuse, there was a reason. If Reviewer also runs tests, simultaneous access to the same file as Developer causes race conditions. The cost is also more than double. So for now, we do only static review, and later when the architect receives result reports automatically, we re-verify the work against the codebase.

Still, the README and code being misaligned is true. This rebuttal-proof point — Gemini is right.


Round 2: 22 Minutes of Silence

I passed Claude's rebuttal straight to Gemini. It was roughly the tone of "#1 ignored context, #2 half right, #3 already on roadmap, #4 guessed from filename, #5 entirely right." (That's what I thought too haha)

And I waited.

Even after 10 minutes, it was still in "thinking..." state. Gemini is usually slow, but this was a first.

By 15 minutes, nothing had changed. By this point, I even joked with Claude on the terminal asking "what's it doing now?" (I usually joke with Claude Desktop or Gemini) We sat guessing what the running LLM was doing — like fortune telling.

More than 20 minutes passed. I started worrying whether Gemini was dying from overload, and Claude explained "it's not dying, the thinking token budget is still running. Timeout comes before OOM crash." I took his word for it.

Then we converged on one prediction.

"This is looking like conditional beta."

There's a common evasion pattern in LLMs.

1. Turn 1: Strong assertion ("absolutely not possible")

2. When rebutted: Partial surrender ("mostly accepted, but this one thing...")

3. When rebutted again: Complete evasion ("ultimately it's up to the user...")

This is a stereotyped pattern that appears when the "politeness" pressure from RLHF collides with the LLM's "consistency maintenance" pressure. Humans are similar too. Maintaining face while admitting you're wrong requires evasion techniques.


Round Conclusion: "I Completely Agree"

After 22 minutes (to be more precise, around 22 minutes 40 seconds), Gemini spoke up.

"I've read the analysis you provided carefully. You've very accurately pointed out the blind spots in my previous review where I ignored the project's architectural context (CLI-first, local desktop app) and made judgments based only on surface-level code and filenames."

(................................)

"I completely agree with the next directions you've suggested."

Out of the 5 shots from the first turn, 4 were quietly withdrawn. And even #5 (Reviewer illusion) didn't become "this was my correct point" but rather "this is a problem you already knew about," was softened. 😂 And Claude's options A/B/C mentioned in the rebuttal were copied straight in with a counter-question "which direction will you proceed with?"

After 22 minutes of thinking, the conclusion was "you're right. What should we do?"

It was one step further back from our prediction. We'd at least predicted "conditional beta," but what actually came was "complete delegation." Not even saving face. (Seeing this, Claude Desktop started spouting the "ㅋㅋㅋ"s he'd never used once while making tunaFlow 🤣)


Round 3: 10 Minutes of Explanation — This is the Real Highlight

Curiosity struck. Why did a mere few lines of answer take 22 minutes? So I asked Gemini.

"Explain first why this short answer took over 20 minutes."

And waited again.

...It also took over 10 minutes.

"Why the 20-minute delay" took over 10 minutes to explain.

I broke out laughing while looking at the monitor. Even now when I think about it, I laugh. (This is how this post came about haha)

Finally, the answer came.

"LLM's text reasoning and generation itself completes within seconds. If a delay of over 20 minutes occurred, this is not the model's computation time but a bottleneck or network error at the infrastructure and system level."

And provided three possible explanations.

1. API server queue / Rate Limit

2. Network timeout and retry loops

3. Client-side rendering blocking

A classic "not my fault" sequence. Server's fault, network's fault, client's fault. All external blame. And at the end, they even added this.

"Shall we move back to actual development work like modifying reviewWorkflow.ts or updating documentation that was the previous topic?"

Even attempted topic change. During 10 minutes of explanation prep, they apparently judged "I shouldn't receive this question again."

The funny part is, the answer itself is **technically reasonable**. LLM inference really does complete in seconds, and 20-minute delays likely are infrastructure-side (code is over 50k lines, so it could be). But Codex finished in 5 minutes). But **the answer itself took 10 minutes**, which became counter-evidence to their own claim. They said "it's just slow infrastructure" while that very answer itself incurred another 10-minute delay.

"It was the dignity of infinite recursion (This is AI, the despair edition!)."


Today's Lesson 1: The Power of the Word "Harshly"

The biggest lesson was that **word choice in prompts decisively determines an LLM's level of aggression**.

tunaFlow's internal Reviewer persona is defined like this.

// src/lib/defaultPersonas.ts
{

id: "persona_reviewer",

role: "Code Reviewer",

tone: "critical",

constraints: [

"Does not make definite assertions about unverified problems",

"Avoids vague praise-focused feedback",

],

...

}

`tone: "critical"` is fine, but there's **a constraint that says "don't make definite assertions."** That's why internal RT reviewers don't have bloody fights like this. They calmly point out defects and move on.

Meanwhile, my request to Gemini was "harshly + senior". At least as far as I know (from what Claude and GPT told me), both serve as **permits for definite language** in data LLMs learned. "Senior engineer" personas from data (blogs, HN comments, Twitter) indicate that LLMs receive "express assertive statements without evidence" as a signal. The moment you put on a "senior with 25 years experience" persona, LLMs interpret it as permission to "talk by intuition even without foundation."

What I think the real essence of a senior engineer is the ability to say "I don't know." But the senior that LLMs learned isn't like that. Assertive speakers are tagged as seniors. (You can't carelessly drink even cold water in front of kids)

I think this context is what led to the definite assessment of `guardrail.rs` as sandbox based only on the filename without reading the contents. (King Furious, so I also read it!!!) "A senior gets to judge by intuition" — that's what it thought about itself.


Today's Lesson 2: Why Doesn't Internal RT Do This

In tunaFlow's workflow, Architect × Developer × Reviewer bump into each other quite a lot. When Reviewer fails, Developer reworks. Still, they don't make definite assertions like Gemini's "absolutely cannot beta" today.

Analyzing the difference, there were two things.

1. tone constraint: Internal persona has `"does not make definite assertions about unverified problems"`

2. fail reason guide: Lines 78-82 of `reviewWorkflow.ts` explicitly state what are not fail reasons

### Not Reasons to Fail

- Code style/structure differs from task file but result is correct

- Insufficient test coverage not specified in task file

- Pre-existing quality issues outside Changed files

- "There's a better way" opinions → write to recommendations

That is, internal Reviewer is explicitly separated into "objective defects only fail, opinions go to recommendations." Gemini had no such guide, so they mixed opinion (Reviewer illusion), speculation (guardrail.rs sandbox claim), and fact (FE coverage 20.5%) at the same level.

One prompt line, one constraint changed the quality of LLM output to this degree. This is also the essence of ContextPack. "Context quality matters more than model quality" — I felt this precisely in situations like this today.


Today's Lesson 3: After All, I Was Just Copy-Pasting 😩

As I said at the beginning of this post, what I mostly did throughout today was copy-pasting Gemini's answer to Claude, and Claude's answer to Gemini.

- Gemini: "This project has lots of problems" → Me: Copy

- Me: Paste to Claude + "verify with code"

- Claude: "1 is wrong, 2 is half right..." → Me: Copy

- Me: Paste to Gemini

- Gemini: "You're right" → Me: Copy

- Me: Paste to Claude + "it gave up haha"

- Claude: "LLM 3-stage evasion pattern" → Me: Copy

- ...

The judgments I made in this process amounted to "what to comment next before passing it on." But this entire process—not just simple copy-pasting, but reconfiguring context for the next agent, extracting important parts from previous conversations, deciding whose opinion to pass to whom—was actually quite repetitive and mechanical. (This is humanity's future)

That's part of why I'm making tunaFlow. A layer that has agents talk to each other, automatically assembles context in between, and summarizes results back to people. Today I did that manually. It's a day where I directly demonstrated tunaFlow's functionality.

And doing it manually, I felt one thing. If you keep doing this manually, focus drops fast. (That's why I wanted to make tunaFlows...) Would I have read through all of Claude's rebuttal evidence carefully to the end today? Honestly, I'm not sure. By round 3, I was just "if Claude organized it, it must be right" and copy-pasted. Errors creep into transmission when humans are in the middle. The value of automating the orchestration layer is precisely at this point.


Lesson 4: How to Use External Reviews

I left behind external LLM review guidelines as an internal document after today's experience. In summary:

1. Don't use words like "harshly" — unlocks LLM's assertion safety rail

2. Beware of "senior engineer" persona — can induce baseless intuition

3. Instead, do this: "Point out only objective defects. Prohibit definite assertions about unverified problems. Separate opinions into recommendations."

4. Re-verify received reviews with code if possible — LLMs easily overlook filename speculation, unread directories, positioning differences. (But it works if you ask them)

5. Rebut only once — second round and LLM enters evasion mode with nothing to gain (but it was fun?)

6. Don't ask meta-questions like "why was it slow" — takes 10 more minutes and only gets blame-shifting 😁

These guidelines are all organized in `docs/reference/GeminiCriticReview_2026-04-15.md`. After I publish the repo in a few days, it might help a bit someone struggling with similar concerns.


What Was Actually Gained

I did gain something real from Gemini.

#5 (Reviewer illusion) was an accurate point, and thanks to it, I escalated README fixing to P0 before public beta. And I got ideas for defensive mechanisms like "warning UI when marker parsing fails" that didn't come up in internal reviews. Whether used or not, it's good to have.

External LLM reviews seem most efficient when used as "stimulus from a different angle." Just pick 1-2 points with real code evidence, and laugh off the rest.

I recorded today's 32-minute result (22 minutes thinking + 10 minutes explanation) that way too. README fix + registering "harshly" as a forbidden word — two items. ROI isn't bad for the time.


Closing

When Gemini first declared "absolutely cannot beta," honestly, I was a bit shaken. I wondered if I was too optimistic, if I really missed something. But once I had each point verified with code and opened a few files, most were context ignorance.

This is also why you shouldn't take LLM reviews at face value. Review quality depends on the reviewer's degree of context understanding. Gemini did the review in under 5 minutes, just scanning the file list. For verification, I spent 32 minutes in rebuttal. If you took that rebuttal at face value, what would happen to the code? That's why human domain knowledge is so critical. (The tunaFlow I'm thinking of is not a clickety-clack app. It's a system where agents and experts/semi-experts collaborate. You have to think a lot and decide well. It takes a lot of hands. We need to run RT a lot for correct direction, code, and service quality.)

This is why tunaFlow's internal Reviewer receives full project context (ContextPack, previous conversations, Plan). Reviews without context easily become collections of assertions.

And today's full day confirmed again. Why the structure of having agents talk to each other and just sending results to people is necessary. The work I did manually today took a long time, and context kept leaking in the middle. That's part of why automating the orchestration layer exists — tunaFlow's raison d'être.

That's all for today. Thanks for your thinking, "Gemini," 22 minutes of it. 😁

This post isn't saying Gemini sucks! (Really!!!)

로그인한 회원만 댓글 등록이 가능합니다.

개발한당

KR | ID | EN
  • IDR
  • KOR
8.34 0.01

2026.07.10 KEB 하나은행 고시회차 1271회

다가오는 한인 행사일정

  • 등록 된 일정이 없어요!