After nearly a week of effort, it seems the llm wiki system has crossed a major milestone.

I first encountered it through secall by a passing developer, and while I've gone through many trial-and-errors and difficulties, it looks like we're at a point where just a bit more refinement could get us to full automation.

The content below is a summary of the llm wiki that Claude and I have built together.

🏗️ Vault Knowledge System Summary

It started from Karpathy's LLM Wiki pattern, but has evolved with fundamentally different sources and philosophy. We reference the LLM Wiki pattern as conceptual background, and this document records our system's unique philosophy and current state.

📋 Table of Contents

• Core Philosophy — Work Itself is Knowledge

• Differences from Karpathy Original

• Pipeline — User → Session → Knowledge

• Advances

• Shortcomings / Challenges

• Structural Principles

• Session Continuity — Handoff

Core Philosophy — Work Itself is Knowledge

Karpathy's original source is external documents (articles, URLs, books).
This system's source is your own AI session logs — the process of working itself becomes raw data.

Karpathy: external world knowledge → organize for me
This system: my working process → itself becomes knowledge

A self-referential knowledge system

Vault is not a container for external information, but a space where my thought process accumulates. Knowledge automatically builds up at the end of each session, and I can converse with my past self.

Differences from Karpathy Original

Pipeline — User → Session → Knowledge

This system's entire data flow is organized in three layers.

┌─────────────────────────────────────────────────────────┐
│  User                                                   │
│  Work with Claude Code / Claude AI                     │
└────────────────────┬────────────────────────────────────┘
                     │ Auto-capture during work
                     ▼
┌─────────────────────────────────────────────────────────┐
│  Session                                                 │
│  raw/sessions/YYYY-MM-DD/{agent}_{project}_{id8}.md     │
│  secall: BGE-M3 vector embedding → SQLite + ANN index   │
└────────────────────┬────────────────────────────────────┘
                     │ Gemma4 cron (09:15 / 15:00 / 21:00)
                     │ Claude PR review
                     ▼
┌─────────────────────────────────────────────────────────┐
│  Wiki                                                    │
│  wiki/projects/ · wiki/topics/ · wiki/decisions/        │
│  → Context feedback to next session on recall            │
└─────────────────────────────────────────────────────────┘

Layer 1 — User

Users work through two channels: Claude Code (coding/operations) and Claude AI (exploration/conversation).
Even across different channels, sessions are collected uniformly and distinguished by the agent field.

Project context: Sessions auto-extract project names from cwd (working directory).
614 sessions, 114 projects — the working directory itself becomes a knowledge category.

Layer 2 — Session

Session lifecycle is automatically managed by secall.

Indexing structure:

Session file (Markdown)
    ↓ chunk splitting
BGE-M3 (1024-dimensional vectors)
    ↓
┌──────────────────┬────────────────────────────┐
│  SQLite turn_vectors table  │  ANN (USearch) index  │
│  + FTS5 full-text search index │  approximate nearest neighbor search │
└──────────────────┴────────────────────────────┘

Search query types:

Layer 3 — Knowledge (Wiki)

Sessions become wiki through two paths.

[Automatic] Gemma4 cron (09:15 / 15:00 / 21:00)
    Session → wiki/pr/{id8}.md (first draft)
        ↓ A: inject existing wiki context
        ↓ C: duplicate detection (80%+ → 3-second SKIP)
    Claude PR review
        ↓ merge / create / discard decision
    wiki/topics/ · wiki/projects/ · wiki/decisions/

[Manual] Claude /wiki · /wiki-batch · /wiki-query
    Ingest or synthesize directly at needed points

Knowledge feedback: Wiki is not a static repository.

User question
    ↓ wiki_search (lightweight) or /wiki-query (full exploration)
Related wiki pages + session recall
    ↓ reflected in Claude response
Better work results → new session → better wiki

Project-level knowledge isolation

wiki/projects/ maintains current project status independently.
Project A's decisions don't mix with project B through the sources: frontmatter tracking.

Advances

1. Multi-LLM Role Division

Speed and accuracy have been separated.

Karpathy's original uses a single LLM for everything → can't satisfy both quality and speed simultaneously.

2. Two-Tier Validation Pipeline

Drafts created by Gemma4 are reviewed by Claude against existing wiki, then judged (merge/create/discard).
Claude supplements the structural limitations of the smaller model (classification errors, duplicate creation).

→ Wiki PR review two-tier validation system

3. PR History Feedback Loop (2026-04-15)

Completed PRs are not deleted but archived to wiki/pr/history/YYYY-MM-DD-{id8}.md.
Decision reasons are structured via the verdict field in frontmatter and ^review blocks.

Purpose: accumulate judgment history
   → Statistics: Gemma4 accuracy, discard reason frequency
   → Future judgment: "how was this type judged before?" recall possible
   → Gemma4 tuning: recurring misjudgment patterns → prompt improvement basis

This creates a self-improving system loop beyond simple record-keeping.

4. Lazy Refinement Loop

Sessions missed by wiki-execute don't disappear.
Preserved in raw/sessions/ → discovered at needed moments via /wiki-query recall fallback → promoted to wiki.

Automatic processing (speed-first, context-limited)
    ↓ missed sessions remain in raw
Demand-driven discovery (question-triggered, context clear)
    ↓
More accurate wiki page generation

"Wiki should be created when need is clear" — better signal-to-noise ratio than automatic recommendations.

5. A+C Quality Improvement

Sessions with 80%+ duplication to existing wiki SKIP in ~3 seconds instead of ~70 minutes of generation.
Requirement: secall serve (port 8765) running.

6. Session Continuity — Handoff (2026-04-15)

Fundamental LLM limitation: context disappears when session ends.
This system overcomes this by generating a handoff file at session end, passing it to the next session.

Session end (/session-close)
    ↓
generate share/project/obsidian-vault/handoff/YYYY-MMDDThhmm.md
    ↓
Claude auto-reads at next session start
    ↓
understand previous completed work + planned work + current status

Handoff file structure:

Why use Obsidian CLI:

# operate with vault relative path → same path even if PC changes
obsidian files folder="share/project/obsidian-vault/handoff"
obsidian read path="share/project/obsidian-vault/handoff/"

Unlike ls + Read tool (absolute paths), Obsidian CLI uses vault-root-relative paths.
Works identically even moving between MacBook ↔ NAS ↔ different PC.

Handoff is externalized LLM memory

Not filling gaps between sessions, but vault itself becoming persistent memory.
"Past me → Future me" — a status summary sent to yourself.

7. Session Lifecycle Awareness

Sessions re-entered with --resume undergo partial reprocessing if turns_seen < total_turns.
Tracks same-session growth, with PR history filenames auto-distinguishing by date.

Shortcomings / Challenges

❶ Paragraph-level Source Tracking (Not Implemented)

Currently only page-level sources: frontmatter exists.
No claim-level source tracking like ^[session_id] paragraph marks in llm-wiki-compiler.

# Future goal example
Gemma4 supports word-level timestamp. ^[9c979986]

Unnecessary when pages are few. Consider adopting when hundreds of pages + multiple mixed sources.

❷ Large Session Automation Incomplete (100+ turns)

Sessions over 100 turns run manually with wiki-execute-large.sh. RTX 5080 (NAS) recommended.
Excluded from cron automation.

❸ Administrator Bias (Improving)

Claude tends to excessively discard based on project/topic classification criteria.
Will improve as PR history accumulation enables data-driven pattern analysis.

❹ exportignore Flexibility Lacking

Settings to exclude specific sessions/projects from indexing are hardcoded in SQLite queries.
The .exportignore file approach in llm-wiki-template is more flexible, but requires secall Rust code modification.

❺ No Confidential Session Handling

No encryption mechanism for storing health data or personal finance-related sessions.
Can reference llm-wiki-template's sessions/confidential/ + GPG approach.

❻ Dataview Statistics Dashboard (Phase 4)

No Dataview dashboard visualizing verdict fields in wiki/pr/history, wiki page count, source distribution, etc.
Foundation data (frontmatter) is already being accumulated.

Structural Principles

Source Layer Immutability

Files under raw/ are never modified. The observation target must not change.

Wiki Focuses on Current State

Wiki contains current state, not history.
Records and decision-making are separated into decisions/.

Agent Ownership Separation

Obsidian CLI Priority

Access vault files via Obsidian CLI. Works with vault-relative paths, so paths remain identical across different PCs.

Session Continuity — Handoff

→ See "7. Session Continuity — Handoff" in Advances section.

Key summary:

• /session-close skill generates file in share/project/obsidian-vault/handoff/

• Before Claude's first response in next session, automatically checks latest file via Obsidian CLI

• Stores completed work / planned work / system status in structured form

• Obsidian CLI-based → vault relative paths → PC portability secured