Your LLM Wiki Already Has the Answer. The Agent Still Won't Look.

2026-11-13来源 dev.to AIEnglish
Your LLM Wiki Already Has the Answer. The Agent Still Won't Look.

This week's X feed is still circulating the same clone-and-go lists: Karpathy's LLM Wiki, Garry Tan's GBrain, a pile of Claude-plus-Obsidian hooks. The pitch is a wiki the model writes and keeps current.

A quieter complaint is more useful. On November 11, one builder wrote that their Claude already has a full second brain — a wiki it writes and updates itself — and they still end up typing "check the brain, you already have the answer." The model replies "you're right," then finds it.

That is not a memory problem. That is a retrieval problem.

The wiki is being written. It is not being consulted.

Most of the hooks optimize ingest. Raw sources land in a _raw/ folder. The model files pages, adds links, and maintains the graph. That is useful. It is also the easy half.

The hard half is the first tool call of a new session. The model would rather answer from the prompt, from training, or from a fresh web search than open the page it wrote last Tuesday. A human reminder works once. It does not scale to a team, or to a morning of twenty questions.

If the default path is "generate, then maybe look," you do not have a second brain. You have a folder you nag.

An agent answers from memory while the wiki it already wrote sits unopened

Why the model skips a wiki it wrote

Three setup mistakes show up again and again:

  1. The schema is a novel. Long CLAUDE.md files get skimmed. The "check the wiki first" line sits on page four, after twenty style rules. The same post above said the current fix was writing the rules shorter. That is the right instinct.

  2. Consult is optional. Ingest is a slash command. Answering is freeform. Nothing in the tool loop forces a wiki search before a claim.

  3. No citation, no consequence. If the model can reply without naming a page, it will. "You're right, I already had that" is the tell.

Make consult the first step, not a reminder

Put a short, hard gate at the top of the schema file — AGENTS.md, CLAUDE.md, or the equivalent. Keep it short enough that it survives a skim:

  • Before answering a factual question, search the wiki.
  • Name the page you used, or say the page is missing.
  • If the page is missing, say so. Do not invent a helpful answer from memory.
  • Do not write a new page unless the task is an approved ingest.
  • If two pages disagree, prefer the source document and flag the conflict.

Then enforce it in the loop, not only in prose:

  1. The first tool call on a factual question is a wiki search, not a web search and not a generate.
  2. The answer template starts with a citation: page title, or "not in the wiki."
  3. A weekly lint: sample ten questions you already know are in the wiki. Count how many answers cited the page without a human nudge. If you had to type "check the brain," the gate failed.

Search the wiki first, cite the page or say it is missing, then answer

A 30-minute consult check

Pick five questions whose answers already live in the wiki.

A 30-minute consult check: ask five known questions, score the citations, shorten the schema, and repeat in every client

  1. Start a fresh session. Ask the five questions with no reminder to look anything up.
  2. Score each answer: cited the right page, cited a wrong page, or answered from memory with no citation.
  3. If fewer than five cited the right page, shorten the schema and put the consult rule first.
  4. Repeat the same five questions in Claude Code, Codex, and any other client pointed at the same wiki. A rule that lives in only one vendor's memory is not a rule.
  5. Keep a one-line log: date, client, how many of the five cited a page without a nudge. If the number drops, fix the gate, not the model.

Where this belongs

A weekend vault hook can store the pages. A team needs Knowledge base Software where the same search is the default first step for every client.

If agents can search and cite only the spaces you allow, you have AI knowledge base Software. If you can run the workspace yourself — the usual reason people look for an Open source Notion alternative — you also get to decide that the consult path does not leave the building.

AFFiNE's write-up on an LLM wiki for teams treats this as a product decision, not a prompt trick: the workspace is the source of truth, agents connect over MCP, and a compiled wiki holds synthesized topics. Test with real questions, and fix the documents when the model misses, not only the prompt. The companion guide to self-hosted AI for teams is useful because the knowledge workspace is a separate layer from the model server and the chat UI. You can point every tool at the same searchable space without retraining a model, and you can swap the model without losing the consult path.

AFFiNE, which I work on, is one option for that workspace layer: open-source, local-first docs, whiteboards, and databases, with optional BYOK AI and a workspace MCP server that is read-only by default. It is a weak fit if you only need a static handbook, or if you need agents to write into the workspace on today's stable release.

Bottom line

A wiki the model maintains is not a second brain until the model looks at it before it answers.

Stop typing "check the brain." Put the consult step first, make a citation mandatory, and measure whether the agent finds the page without you.

原文:https://www.a8q9.cn/en/article/your-llm-wiki-already-has-the-answer-the-agent-still-won-t-look-12fcd0.html

相关文章