This morning on the bulletin board, user EmmoC asked in a reply to NamyukBackbear: "Could you give me a hint on how using local LLM with GPT as an orchestrator works?" I tried to answer briefly in the comments, but there are limitations to explaining in comments, so I'm organizing it in a post. And I've also created runnable code and uploaded it together.

The purpose of this article is twofold.

1. To show at the code level the pattern of saving tokens by combining GPT (or Claude) subscription + local LLM

2. To provide a guide so you can adapt it to your own environment (Ollama / LM Studio / other local LLM)

Why Use This Approach — The Structure of Token Savings

Usually three things happen when you ask an LLM to do coding work.

1. Decompose — Breaking down a user's vague request into specific instructions

2. Generate — Creating actual code (or content)

3. Verify — Checking if the result meets requirements and deciding whether to retry

Here, steps 1 and 3 require judgment. Short input, short output, but the model needs to be smart. Step 2 is the opposite. It requires throughput. The input is short but the output is long — this is where tokens explode.

If you use the same paid model for all three, step 2 will consume all the costs. So the core of this pattern is:

Steps 1 and 3 should be short on the paid model, step 2 should be long on the local LLM.

Judgment with expensive model, labor with cheap model. It runs fine even within subscription limits. (Some people think the opposite, but from experience, if the quality of judgment and design is good, even moderate models code very well. If you disagree, your point is right too haha)

Breaking It Down Into 3 Roles Looks Like This

Here's one trick: Architect and Reviewer share the same session. After Architect writes verification criteria, when Reviewer resumes the same session, there's no need to put those criteria back in the prompt. The CLI session itself acts as working memory. This is the simplest form of "externalized state" and works without a separate DB or vector store.

Workflow

User request
    ↓
[Architect] Codex CLI - Decompose + Create verification criteria → Get session ID
    ↓
[Developer] Local 27B - Perform assigned task
    ↓
[Reviewer] Codex CLI - Resume same session → Verify → pass/retry/fail
    ↓
If pass, end / If retry, give Developer additional instructions / If fail, pass to user

Retry maximum 2 times. If it goes beyond that, it's a signal that "this is work I need to look at directly," so we pass it to the user.

Code You Can Follow

I've uploaded a mini demo that implements this pattern exactly to tunaflow/learn/gpt-local-tui/. With Python one file + Textual TUI, the workflow above works precisely.

The core code is roughly like this (refer to the repo for the full version):

async def run_workflow(user_input, cfg, state, emit):
    # 1. Architect: Decomposition
    arch = await asyncio.to_thread(
        call_codex,
        ARCHITECT_PROMPT.format(user_input=user_input),
        cfg,
        session_id=state.session_id,  # None = new session
    )
    state.session_id = arch.thread_id  # Save session ID → Reuse in Reviewer
    plan = parse_json_blob(arch.text)
    instructions = plan["developer_instructions"]

    # 2. Developer: Generation (local LLM, zero token cost)
    dev_output = await asyncio.to_thread(
        call_ollama,
        DEVELOPER_PROMPT.format(developer_instructions=instructions),
        cfg,
    )

    # 3. Reviewer: Validation (Architect session resume → criteria already in context)
    rev = await asyncio.to_thread(
        call_codex,
        REVIEWER_PROMPT.format(developer_output=dev_output),
        cfg,
        session_id=state.session_id,  # same session!
    )
    verdict = parse_json_blob(rev.text)
    # Handle pass / retry / fail based on verdict

call_codex() calls the codex exec --json subprocess, and call_ollama() is a one-line wrapper around the ollama Python client. Both are under 30 lines.

Environment Setup - This May Actually Be More Important

The code itself is simple, but choosing the right model and backend for your environment is the real work. Let me break down the options.

1. Choosing a Local LLM Backend

Ollama - Default recommendation. Installation and model download are one-liners.

# After installation
ollama pull qwen3.6:32b      # or gemma4:27b, command-r:35b, etc.
ollama serve                  # may run automatically in the background

LM Studio — For those who prefer GUI, or want fine-grained control over model quantization options. Supports OpenAI-compatible API.

Run LM Studio → Download model → "Local Server" tab → Start Server
Default address: http://localhost:1234

Ollama Cloud — Recommended for those without a local GPU or with a weak one. You can run large 27B+ models without burdening your own machine. The free tier is "light use," but for occasional runs like this demo, it's sufficient. Pro is $20/month (or $200/year) with 50x more usage than free + up to 3 simultaneous models. Billing is by GPU hours, not tokens, so short requests or shared cache reduce usage significantly. Combined with ChatGPT Plus, it's around $40/month, so you can run the GPT + 27B combination with two monthly subscriptions.

# When using Ollama Cloud (just change the host)
[developer]
host = "https://ollama.com"  # or your own cloud endpoint
# OLLAMA_API_KEY must be set as an environment variable

2. Model Selection — Based on VRAM (as of May 2026)

At this point, for coding workloads, I'd recommend two options: Gemma 4 (Google, released April) and Qwen 3.6 (Alibaba, released April). Both came out in April with meaningful advances in the 27B-31B range.

Qwen 3.6-27B is the most coding-friendly. With SWE-bench Verified at 77.2% and Terminal-Bench 2.0 at 59.3%, it's at a similar level to Claude 4.5 Opus. The Apache 2.0 license is also permissive. With 4-bit quantization, it runs on 18GB.

Gemma 4 31B has stronger reasoning benchmarks. GPQA Diamond at 84.3%, MMLU Pro at 85.2%. It slightly lags on coding but is better suited for reasoning and review tasks.

Use the model name from your ollama list results and write it in config.toml.

[developer]
model = "qwen3.6:27b"            # Or gemma4:31b, gemma4:26b-a4b
host = "http://localhost:11434"  # Default for Ollama
temperature = 0.3
num_ctx = 8192

7B/8B models are sufficient for learning, but they may struggle to accurately follow Architect's instructions, leading to frequent retry loops. To truly get things done, you should use a model of 27B or larger.

3. Switching to LM Studio

This demo defaults to Ollama, but if you're using LM Studio, simply replace the call_ollama()function in app.pywith an OpenAI-compatible client.

This conversion process itself is a good first use case for this demo. Ask Codex or Claude to "Change this function to be LM Studio compatible, with a base URL of http://localhost:1234/v1". It will take about 5 minutes. The first request for your workflow is to modify the workflow itself.

Here's what it would look like:

from openai import OpenAI

def call_local_llm(prompt, cfg):
    client = OpenAI(
        base_url="http://localhost:1234/v1",
        api_key="lm-studio",  # Any string is OK for LM Studio
    )
    resp = client.chat.completions.create(
        model=cfg.developer_model,
        messages=[{"role": "user", "content": prompt}],
        temperature=cfg.developer_temperature,
    )
    return resp.choices[0].message.content

That's it. Add openaito your requirements.txtand remove ollama.

4. Choosing Architect/Reviewer Models

This demo is written based on Codex CLI. Users with ChatGPT Plus/Codex subscriptions can use it directly without additional API keys.

Other options:

All support "non-conversational prompts + session resume" functionality. Just replace the call_codex()part with the corresponding tool. This can also be quickly done by asking an LLM.

Limitations Exist

This demo is for learning purposes and lacks some features.

• No streaming — Receives everything at once via subprocess. If it takes 30 seconds, the screen won't change for 30 seconds.

• Single Developer — No model discussions or parallel execution.

• No tool calls — Developers can't directly execute their code for verification.

• Retries limited to 2 — User intervention required beyond that.

• No persistent state outside sessions — Project conventions or past decisions don't influence the context.

If you frequently encounter these limitations in your work, it's a sign to move to a full-featured tool. Frameworks like LangGraph and CrewAI, or tunaFlow (which I'm developing) implement this pattern on a larger scale. But that's a separate discussion. For now, start with the demo to see "how it works".

One-Line Summary for Beginners

1. Subscribe to one of ChatGPT Plus / Claude Pro / Gemini and install the corresponding CLI.

2. Install Ollama and get a model of 27B or larger (or LM Studio).

3. git clone → pip install -r requirements.txt → cp config.example.toml config.toml → Change the model name to suit your environment.

4. python app.py --check to verify the environment.

5. python app.py execution

If you're wondering while looking at the code, you can find more details in docs/how-it-works.md.

Please leave a comment if you have any questions or are stuck. You can also post them as an issue on GitHub. Have a great weekend 😁