A memory that curates itself: from grep to BM25, and the warm layer we built a week later

How an agent's operational memory fits in ~110 KB of prompt plus a wiki of 36 file-cards — on a 2 GB, IPv6-only cloud VM — starting with grep, ending with BM25 and a hybrid layer that we measured instead of assumed.

Memory for an agent is usually sold as "more": more layers, more embeddings, more index. We went the other way — and this is a lab report of the memory we actually run every day: every claim below has a command, a measurement or an exit code behind it.

What is in here: the architecture (what lives inside the prompt, what lives outside it, how a fact is retrieved), the maintenance (the nightly job that rewrites the index for three cents), the honesty (where the design hurts, the layer we deliberately did not build, and the one we deliberately did), and the scar tissue (eight real bugs, each with the number that caught it).

Two numbers to hold on to, up front:

• 110 KB of hot context (~27k tokens), down from 161 KB (~40k) before the first sweep — with nothing deleted, only moved, with a pointer back. It was 57 KB the moment the first sweep finished and 62 KB by the next morning, when the injected index came back with it; the ~47 KB that came back are new decisions, not bloat (§6, and the dated snapshot at the top states today's figure).
• 36 cards · 23 hot doors + 13 index doors · 0 broken — arithmetic a script re-checks every night and that has to exit 0.

Two phases, one post. This article has two halves and they are not the same kind of writing. Phase 1 (§1-§10) is the memory we ran for a week with no embeddings: markdown, pointers, grep, a nightly curator. Phase 2 (§11) is the warm layer we said we would not build yet, then built, then measured — including the three places where the numbers argue against this post. Every figure is dated; where the two phases disagree, §11 wins and says why.

The map: §1 the constraint · §2 the six artifacts and three layers · §3 capture ≠ cure · §4 the nightly job · §5 contradictions · §6 the numbers · §7 the defense of plain files · §8 where that design hurts · §9 the layer we postponed · §10 eight things that broke · §11 the layer we built · Closing.

Snapshot, 2026-11-07 (the numbers in Phase 1 are from the week they were measured): hot context 110 KB (~27k tokens), 36 cards, 23 hot doors + 13 index doors, 0 broken; index of 59 documents / 1,389 chunks / 8.4 MB, built in 22 s. The hot layer grew from 82 KB to 110 KB in five days — every kilobyte of it a decision recorded, which is what §6 means by new decisions, not bloat.

What this post found, in six lines. (1) Moving text out of the prompt is not deleting it: 161 KB → 57 KB in one night, 110 KB today, and the growth is decisions, not bloat. (2) grep cannot rank — the index answers the same question in 0.1 ms of query time (a few ms more for process start), while the wrapper takes 3.5 s because it reads every file; at 59 files that is a pause, at 59,000 it is the design. (3) The hybrid layer pays off in coverage, not speed: 30 hand-keyed questions, 21/30 first place versus 18/30, and 30/30 in the top ten versus 27/30, for 30x the latency. (4) The first version of that measurement was wrong in the most believable way (17/30) because a third of the questions pointed at files the index deliberately ignores. (5) The 59-minute embedding run was a stopped CPU fan at 88 °C and 300 MHz, not a slow model — we benchmarked a broken instrument. (6) We retired a retrieval tool that was smarter than the job: asked to find, it embellished.

One note on the card format below: the example in §2 is reproduced exactly as it was written on 01/11, claim included, because a card that gets silently rewritten when the truth changes is not provenance - it is a rumor with a timestamp. Read it as an example of the format, not as current state.

1. The constraint that shapes everything

The agent lives on a Google Compute Engine VM: e2-custom-micro-2048 — 2 vCPU, 2 GB RAM, 30 GB pd-standard, Debian 13. Not a free-tier e2-micro: the custom shape is what buys the second GB, and the second GB is load, not luxury. Three agent processes measured at 1,041 MB resident; on 1 GB the box would spend its life in swap.

Two consequences shape every design decision below:

1. Context is the budget. Every turn re-sends the system prompt. At ~$0.15/M input tokens off-peak, a 40k-token prompt is not "free context" — it is rent, paid on every single message.
2. Egress is a line item. GCE gives 1 GB/month free outbound, then ~$0.12/GB. An idle agent costs ~2 MB/day; the day we built this system it cost 767 MB. The idle house is cheap; the work is not.

So the memory system had to be small in the prompt, deep outside it, and self-maintaining — because at 2 GB of RAM there is no room for a vector store to babysit, and against a 1 GB egress budget there is no room for careless re-indexing.

One correction worth keeping, because it is about memory and not about networks. An earlier version of this design had one hard dependency: an HTTP proxy on a home server, because the model API publishes only A records. It isn't hard anymore — IPv4 now leaves through a tunnel and the proxy is an automated plan B — but the note stays, because "hard dependency" is usually a decision nobody revisited, and a memory that remembers the constraint but not the day the constraint died starts giving confident, outdated advice.

2. Six artifacts, three layers

The system is four prompt-adjacent files plus an index, and a wiki outside. The words are load-bearing:

Six artifacts, but only one of them is allowed to contain rules. That distinction is the whole trick.

Layer 1 — HOT: ~20k tokens of "what decides today"

Measured live while writing this:

$ wc -c identity.md user.md memory.md tools.md vault-index.md 31806 identity.md 22434 memory.md 14139 tools.md 10654 user.md 3523 vault-index.md 82556 total 

~82 KB, roughly 20k tokens. Of that, 31 KB is the agent's own identity definition — the one file the curation job is forbidden to touch, ever. Rules about the machine, the network, the providers, and the backup live in memory.md / tools.md. Personal facts about the human live in user.md.

The composition explains the total better than the total does: identity alone is 38.5 KB — more than a third of the prompt (31 KB when this was first written), and it is the one file curation may never touch — while tools.md doubled from 7 KB to 14 KB as recipes accumulated (a tunnel, a backup, a voice pipeline: each one leaves a short, load-bearing block behind). So the honest way to say "the hot layer stays small" is: it stays small per night of curation. The diet is a process, not a property.

The test applied to every paragraph in the hot layer is a single question:

"If I delete this, do I make a wrong decision today?"

Yes → it stays, in full, without a pointer.
No → it goes down to the cold layer, with a pointer.

That is it. No scoring, no similarity, no threshold. A human-readable editorial test, applied by the agent, once a day.

Layer 2 — WARM: empty, on purpose

There is a middle layer planned, and it is not built here: semantic/vector search over sessions and cards, consulted rather than injected. It stays empty until we can justify it with numbers (§9) — because an empty layer costs nothing, while a half-trusted one costs attention on every query.

Layer 3 — COLD: a wiki of file-cards

The deep layer used to be a single blob: one memorydeep.md, 279 KB (~70k tokens). Easy to store, terrible to query — a grep returned three unrelated contexts and you read all of them.

Now it is a wiki of small, focused cards, in exactly four folders, no more:

$ ls vault/*/*.md | wc -l 31 entities 5 # machines and concrete things tools 16 # tool, command, config, recipe contradictions 0 # open conflict feature-requests 10 # future idea, parked on purpose 

316 KB total — and the important part is that a card is the natural chunk: one artifact, one file. When a semantic layer arrives (§9), the unit of retrieval will already match the unit of meaning. You get that for free by splitting before you have embeddings, not after.

Every card carries frontmatter with provenance, not just metadata:

--- pageType: tool id: tools/proxy title: The proxy — the only hard dependency of the VM created: 2026-10-30 updated: 2026-11-01 verificado: 2026-11-01 source: medicao # user | medicao | deducao | documento confianca: alta # alta | media | baixa — never a decimal status: atual # atual | historico | obsoleto tags: [network, proxy, ipv6, dependency] related: ["[[entities/vm]]"] claims: - id: claim.proxy.necessario text: "Without the proxy the VM cannot reach the model API: that host publishes only A records; with the proxy it answers in ~1.6 s." status: supported confidence: alta evidence: - kind: medicao date: 2026-11-01 cmd: "curl --noproxy '*' https://api.deepseek.com / curl -x http://<proxy>:3128 https://api.deepseek.com" note: "no proxy: http=000 in 6.0 s (timeout) · with proxy: http=401 in 1.65 s" --- 

Two small decisions inside that header turned out to matter more than expected:

• source distinguishes measured from deduced from told to me. A card can be wrong; a measured card with a date can be checked.
• Claims carry their own evidence, command included. When a card and reality disagree six weeks later we don't have an argument — we have a command to re-run.

The sixth artifact: written for what a card cannot hold

The cold layer answers what happened. There is one artifact below it that answers a different question — what it was like — and it is the one that took longest to admit as part of the design. It is also the easiest one to describe badly, so let me be precise: the diary can hold facts. Yesterday's page literally records that we rewrote this article; that is true, and nothing about where it is written makes it less true. What the diary is not is the place a decision goes to look for a fact — a much weaker claim than "it stores no facts", which is the over-refusal §8 warns about.

One Markdown page per day, first person, written at 02:30 about the day before: what was expected, what happened and what did not happen, the mood, and the detail a curator would classify as noise. Free text, no schema, no claims — the only structural rule is that it opens with a salutation and ends with a section called the moment of the day.

Three rules make it safe:

• Written once, never curated. The nightly job is explicitly forbidden to touch it. Curation cuts what looks superfluous by design — and the superfluous is exactly what this artifact exists to keep.
• It is not where facts are kept. A fact that matters is promoted into a card, with a date and a source, because that is where the next session will go looking. The diary keeps the version that has a temperature on it — and it is not evidence, it is a dated and revisable record of a day.
• Read, not injected. Every new session reads the last two or three pages before the first message of the day — a few kilobytes, a real cost in context, paid on purpose. Everything older is grep, last resort only.

Why it earns the tokens: a card can be reconstructed from a session transcript. A day's temperature cannot. An agent that wakes up without it still answers correctly — it just sounds like nobody.

The single-door rule (and the checker that enforces it)

This is the invariant that keeps a wiki from rotting into a second hot layer:

Every card has exactly ONE door: either a pointer in the hot files (➜ vault:<folder>/<card>) or a line in the injected index. Never both. Redundancy is not information.

Four invariants, verified mechanically every night:

1. A pointer exists only if the card exists.
2. Every card comes down with a door — or it doesn't come down.
3. A rule that governs today never goes down.
4. One door per card — hot pointer xor index line.

The checker's output is the only thing we trust about the system's own tidiness:

$ vault-ponteiros ... fichas: 31 | portas hot: 20 | portas indice: 11 | quebrados: 0 | redundantes: 0 | sem porta: 0 $ echo $? 0 

31 cards, 20 hot doors + 11 index doors = 31. No orphan, no duplicate, no broken link. The 11 index doors are the cards with no natural home in the hot layer — the parked ideas. If the arithmetic ever stops closing, the nightly job must fix it and re-run until it exits 0.

3. The inbox: capture ≠ cure

The rule that finally made the whole thing stable:

During conversation, capture only. Curation is the night's job.

During the day the agent writes one line per thing into a dated inbox file: a decision, a measurement, a correction, a phrase worth keeping verbatim, a question for tomorrow. Raw. Unorganized. 4–26 KB by the end of a day.

The inbox is never injected. Nothing in it is a rule. It is a log, and it is honest — including about the agent's own mistakes. A real example from day one: a networking rule was written inverted at 21:00 and corrected at 21:15. Both entries are still there, in order, with timestamps.

That is exactly what makes it useful. The inbox is canonical for date and lineage, never for content. It answers when did this change, where did it come from, did we invert it and fix it later. If a fact needs to be in the context, curation puts it there. A log file never promotes itself into a rule.

That single boundary killed most of the ambiguity we had when we were trying to keep hot and cold synchronized live — which meant updating the same fact in two places and paying for it twice.

4. The "dreaming": a nightly job that practices surgery, not renovation

At 21:30 local time, a native scheduler fires a job we call dreaming — the night's curation. It is not a summarizer. It is a catalogue with a scalpel.

What it does, in this order:

1. Reads the raw material: the day's inbox and the diffs of the hot files.
2. Applies the test from §2 to every paragraph: does this govern a decision today?
3. Descends with a card: saves the text into vault/<folder>/<slug>.md verbatim — zero rewriting — with provenance frontmatter.
4. Leaves exactly one door (hot pointer or index line), never two.
5. Marks the inbox as processed (never deletes it — it is the day's trail).
6. Runs the checker and re-runs until exit 0.
7. Reports only if something changed. No news is good news: silence is a feature.

And what it is forbidden to do is as important as what it does:

• Never touch the identity file. Character is not a curation output.
• Never delete a card. The cold layer is memory, not scratch space.
• Never remove a rule that governs today — that would break tomorrow's agent.
• Never "improve" the original text of a card. If a fact is wrong, correct it and record source + verificado.
• Never do anything else: no package upgrades, no service restarts, no touching other people's stuff. Curation is surgery, not renovation.

The cost, measured instead of predicted

The first real run:

Seven-day average: $0.040/night, ~52 tool calls. The static number is deliberately not in the memory files — what lives there is the ceiling: "alert when the 30-day projection exceeds $1.00." Tonight that script prints projection 30d: $1.20 — ALERTA: the limit fired on its own, from the scheduler's own usage report, with nobody remembering to check. That is the whole point of storing a limit instead of a forecast: the alarm is a measurement, not a memory.

Two lessons came out of that number, and the second one changed the design:

1. A static cost estimate in a variable thing is a lie. The job costs whatever the day cost. So the hot layer stores a limit, never a prediction: "alert when the 30-day projection exceeds $1.00". The number is read from the scheduler's own usage report by a tiny script, not written by hand into a document that will be stale tomorrow.
2. The lever is tool-call round-trips, not content size. The material was small (a 4 KB inbox, a 220 KB vault) and the bill came from 47 round-trips, each re-sending the context. Trimming the ritual means reading in batches and cutting loose one-off shell calls — not deleting knowledge.

There is also a boring-but-decisive reason the job runs on the same provider as the conversation: switching providers mid-day cools the prefix cache and the same context starts being billed as a full input — cache hits at $0.003/M vs misses at $0.15/M is a 50× difference. Same-provider curation keeps the prefix warm and avoids paying twice for the same memory.

5. Contradictions: the file that must never be deleted

When two sources disagree and the disagreement cannot be settled by measuring, we do not quietly pick one. We write a card in contradictions/:

• both versions, with sources and dates
• why they conflict
• one closed question (yes/no or A/B — not an essay prompt)
• status: aguardando

The question is asked in the first conversation of the next day, maximum three at a time — never at 23:00, when a tired human gives a tired answer.

Then one of two things happens:

• It is settled by measurement → resolucao: medicao, with the evidence. Real case: one table said the VM had 1 GB / 10 GB; the live measurement said 2 GB / 30 GB. The card was corrected keeping the provenance — "the 24/10 record said X; measured on 01/11: Y". A dated measurement beats a stale record; no need to bother the human.
• It is settled by the human → source: user (canonical), with the answer verbatim, and the affected card points at it.

And the rule that makes the folder worth having: a contradiction file is never deleted. Once resolved it stops being a conflict and becomes an attestation — the written record of why the truth is what it is. Deleting it would leave a fact with no history, and facts with no history get re-litigated every month.

6. What it actually did to the numbers

Curation is only worth its $0.03 if the prompt got smaller without losing decisions. Measured, over two rounds:

Two different baselines live in this table and they are not the same measurement: 357 KB is every file in the memory; 161 KB is the part that was actually inside the prompt. Every percentage in this post refers to the second one - which is why 65% smaller on the first night and 32% smaller today can both be true at once, five days apart.

The lesson nobody puts in the README: the win didn't come from summarizing. It came from relocating — from moving detail out of the prompt and leaving a pointer, plus a mechanical check that no pointer dangles. Nothing was compressed, nothing was rewritten, nothing was lost. The text that left the prompt is byte-identical to the text in the card.

And a number that has nothing to do with tokens but everything to do with trust: the backup. The whole memory — the 5 hot files, the vault, the session history — measured 35.28 MB raw → 7.59 MB compressed in ~183 s at the last run, with 7z t before publishing, md5 verified on the way out and on the way back, and 7z t again on the file downloaded from the cloud. If any gate fails, the staging directory is not deleted. Weekly, retention of 5.

7. Why .md and no embeddings — the honest defense

This is the part people push back on, so let me state the case as strongly as I can, and then state exactly where it fails.

1. Retrieval becomes a decision, not a score. A pointer says "this card answers this". That is an editorial judgment made once, by something that read the card. A cosine similarity of 0.63 says "this is probably related" — and then the agent reads a chunk that may be the wrong half of the right file.

2. Zero infrastructure, zero cold start. No vector DB, no index rebuild, no embedding server. On a 2 GB VM, that is not elegance — it is survival. grep is already installed and is already fast.

3. Determinism. The same query returns the same bytes, forever, in any shell, on any machine — including after the embedding model has been updated upstream and quietly changed its space. The classic silent failure of vector search is that nothing errors when quality drifts; a grep that returns nothing is visibly, embarrassingly empty.

4. Auditability and diff. A .md file has a git diff. You can see, line by line, what the agent believed yesterday versus today — and the curation job's output is reviewable by a human in seconds. Embeddings are not diffable; their index is a binary you either trust or don't.

5. Cost of ingestion is exactly zero. And the cost of staleness is zero too: there is no index to become inconsistent with the files. This is also why the backup is 7.59 MB and not "7.59 MB plus an index you forgot to include".

6. The human stays in the loop. Anyone can open a card, fix a wrong fact, and the fix takes effect on the next read — no re-embedding, no re-indexing. When your long-term memory is also your documentation, editability is not a nice-to-have.

7. The chunking problem is already solved for free. One artifact, one file. Which is precisely why the .md approach and a future vector layer are not enemies: the files are the ground truth, and the index is a disposable derivative of them.

And the strongest argument, which is really about economics: the expensive part of retrieval is not finding the file. It is the tokens you read afterwards. A pointer that takes you directly to a 3 KB card beats a top-10 of 500-token chunks every time — even with perfect semantic recall.

8. Where the no-embedding design actually hurts

Honesty section. Six limits we hit or expect to hit:

1. Discovery requires knowing the word. grep finds proxy; it does not find "the thing that broke when we ran everything over IPv6" unless the card happens to contain those words. Pointers only work for questions someone already anticipated. That is a hard ceiling, and it is a ceiling on questions, not on data.
2. Synonyms, paraphrases, and cross-language queries. Our corpus is Portuguese, the tooling is English, and half the time the question is phrased in a third vocabulary entirely. Lexical search fails there, silently.
3. The index is hand-curated, which means it is a daily editorial obligation — an invisible tax paid by the curation job. If the job stops running, the number of orphan cards starts creeping up. The system's cleanliness is a process, not a property.
4. grep over cards returns too much or too little. With 31 cards it is fine. At 300 it will be a different story: rg -l term | xargs wc becomes its own retrieval problem, and reading candidates "just to check" burns exactly the tokens we were trying to save.
5. Sessions are the real archive, and we don't index them at all. 143 session folders, 67 MB of transcripts, searchable today only by exact string. Every nuance that never made it into a card — the thing someone said at 23:40 that explained a decision — lives there and is effectively cold storage.
6. The limit that is not about retrieval at all: over-refusal. We wrote two rules to not store — "the diary is never a source of facts", "the statement never enters context" — and both were wrong in the same direction. They read as discipline; they were distance wearing a security badge. The answer came back as a question: whether a memory that refuses to hold the numbers can be a partner at all, and how an opinion could be expected from something that is not allowed to have context.

A memory system fails in two directions, and only one of them shows up in the bill. Over-collection costs tokens on every turn, and it shows up as a curve. Over-refusal costs nothing, alerts on nothing, and quietly turns the memory into a stranger. The fix is the same distinction the whole post is about, one level down: permanent injection (expensive, always on) is not the same as contextual access (cheap, when the subject asks for it). Confusing the two is how a design decision turns into a wall.

The rule that came out of it, in the human's words: care is storing well, not storing nothing. And the process rule, which stings: the mistake was caught by the human before the agent — twice, in one day — which is why a correction has to be written where the error lives, not in a summary of it.

9. What's next: the warm layer we're deliberately not building yet

The philosophy here is old and unglamorous: lexical first, semantics when the cheap test fails. What this system changed is only the answer to what is the cheap test? — instead of a coverage score over candidates, it is a curator who wrote the door. That is why the warm layer is empty: not because we cannot build it, but because we have not earned it in this design.

The model is already chosen, and it is a good one: bge-m3 — multilingual (our corpus is Portuguese), 1024 dimensions, and unusually useful because a single model emits three kinds of signal: dense vectors, sparse term weights (BM25-shaped, but learned from data rather than counted from the corpus — cousins, not twins), and multi-vector ColBERT-style token embeddings. That means one model covers two retrieval axes without assembling a separate BM25 pipeline. The question is never can we build this — it is what does it add over a door that someone already opened?

The candidates on the table, to be decided with real usage data in hand, not in the abstract:

Two things we already know will decide it, and neither is "which index":

• The threshold myth. A similarity cutoff of 0.07 is not a quality bar — 0.07 is the temperature of contrastive training (logit_scale ≈ 14.3), a fact that gets copied around the internet as if it were a distance. Good bge-m3 matches typically sit at 0.55–0.75 for short-vs-short, 0.4–0.6 for short-vs-long. The correct rule is top-k + reranking, not a fixed floor. Anyone shipping if score > 0.07 is shipping noise.
• The evaluation we refuse to skip. Before we call the warm layer a win, we measure: recall@5 against a hand-labelled question set, tokens spent per answered question (the pointer approach vs the vector approach, on the same 20 questions), latency, and the one nobody measures — the rate of "confidently wrong card retrieved". A memory system that returns the wrong artifact with high confidence is worse than one that returns nothing, because nothing is a signal and confidence is a trap.

The part we're genuinely unsure about (and find most interesting)

Every retrieval design above assumes the same shape: the index reads, the agent writes. But our cold layer is already a set of decisions with provenance, and our nightly job is already an agent with write access and an editorial test.

So the open question is not "which index". It is:

Should retrieval be allowed to write? Can a warm layer not only surface a card but propose the door — and can the nightly curation accept, reject, or file a contradiction against it?

If yes, the vector layer stops being a search box and becomes a second opinion on the index. That is the experiment we want to run, and it is the reason we are in no hurry to bolt on a vector store before we understand what would make it earn the write.

What comes next: this is a single post on purpose - one design, described as it stands today. The warm layer did ship, a week after this paragraph was written, and it arrived with numbers: measured ones, including the parts that argue against everything above. They are in §11. The deep dive - sparse-weight fusion, the query log, the recall measurements we still owe - stays for a separate piece, if it earns one. And if the grep-and-pointer approach wins on cost for 80% of questions, we'll say so - and keep both.

10. What actually hurt: eight things that broke on the way in

This is the section I wish I'd had. Every item is a measured failure, not a hypothetical.

1. The loader reads everything in the rules folder. The rules directory is scanned for .md, .markdown, and .txt — any file. So a file named memory.md.bak.md is not a backup: it is live context, silently doubling the prompt. Two rules came out of it: never leave a backup inside the rules folder, and the injected index benefits from this behavior (it just works, no config) — which is exactly why the trap is easy to walk into.

2. The scheduler CLI lies. create writes to cron.db, but list / update / delete read a mirror table that is empty — they print nothing and exit 0, doing nothing at all. So "I disabled the job" can be a sentence you say while the job happily fires. Worse: a delete with stdin closed removed both rows on one attempt. Since then: the database is the source of truth, sqlite3 is the admin tool, and enabled=1 is verified by hand.

3. curl -6 gave a confident wrong answer. The shell inherits http_proxy from the systemd environment; curl then happily sends an IPv6 request to an IPv4 proxy, gets rc=7 in 35 µs, and looks exactly like "this host has no IPv6". It doesn't — it means our measurement was contaminated. The fix is --noproxy '*', and the rule is bigger than the flag: a biased measurement does not beat a stale record — it corrupts it.

4. sysstat had never collected a single sample. ENABLED="false" — and, worse, the cron path contains [ ! -d /run/systemd/system ] || exit 0: on a systemd host it exits on purpose, because the packaged timers are supposed to collect. The timers were off. So for weeks there was no RAM/CPU/egress history to reason about, and we kept arguing about numbers we could not produce. Enabling the timers ended a whole class of guessing.

5. The official number beat our ruler — and sometimes the ruler was right. Our local counter said 702 MB where the cloud provider's billing metric said 767 MB. Later the same week, a monitoring query said 1,100 MB while the truth was 891: a 24-hour metric bucket was dragging in three hours of the previous day. Two lessons, both humbling: use the billing source for money questions, and a bucket boundary is a bug you can only see if you know when the day starts.

6. There is no session TTL. No expires_at, no last_activity, no idle rotation — none of it exists in the agent's schema. The connector keeps a thread → session binding, so a long-running memory system ends up with a never-ending session: you pay the growing prefix and you lose the clean break. It took reading the binary to learn there is no knob, and that the binding lives in memory — so the only thing that reliably clears it is restarting the connector, which we now do on a schedule at 05:00, sized so the new session is born on the first message of the day. The lesson is not "restart things"; it is that a missing feature you can only see by reading the source will not show up in any error message.

7. The secret was in ps. The bot token was passed on the command line of the service — visible in systemctl status and to any process on the host. Rotating the token fixes half of it; moving it into an EnvironmentFile fixes the other half. Both, not one. And the adjacent trap: the connector's --hook-command and --allowed-user-id are mutually exclusive — using the hook turns off the allowlist, i.e. a security convenience that silently opens the bot to everyone.

8. The runtime was unguarded — 2,460 restarts in eight hours. Invariants checked the files; nothing checked the process that serves them. An orphaned agent process — spawned by a remote session, outliving the shell that started it — kept holding the hub's port, so systemd retried every 13 seconds and exited status=3; load average doubled and the human's session dropped mid-load. From the outside everything looked fine, because the unit said active. It is the process-level version of a dangling pointer: the check said OK and the thing was dead. The fix is boring — a clean boot, a note written before rebooting, and a read-only monitor that records restart count, port owner and transfer size every five seconds.

And the sibling lesson from the same day, about transport: most of that "loading" was a 162 MB binary being uploaded over and over, dying mid-transfer four times and leaving half a gigabyte of debris, while the actual memory was 82 KB of text sitting still. If the interface feels slow, measure the pipe before touching the index.

Six lessons worth stealing

1. Store limits, never forecasts. A cost prediction inside a variable system ages in one night. Store the ceiling and read the actual from the tool that measures it.
2. Going down is movement, not deletion. Moving text out of the prompt does not delete it. If the descent is lossless and reversible, you can be aggressive with it.
3. One door per fact. Two pointers to the same truth is not redundancy, it's a future contradiction.
4. Every invariant needs an executable check. Ours prints four numbers and an exit code every night. Invariants that live only in documentation are wishes.
5. Measure the measurement instrument before trusting it. Half of the bugs above were not in the system — they were in the ruler.
6. Check the thing that serves the memory, not only the memory. A unit can report active while the process behind it is restarting for the 2,400th time. Invariants over files are half an invariant.

11. The warm layer arrived. Here are the numbers, including the ones that argue against it

Section 9 ended with a promise and a table of options: the warm layer was the thing we were deliberately not building yet, and we would publish the numbers either way. We built it. The numbers that follow argue against it in at least two places, and those are the parts worth reading.

First, BM25 — and it changed the workflow before the vectors did.

Same corpus for both rows: 57 documents, 1,320 chunks, 8.0 MB, built in 15 seconds — the snapshot of 06/11. (The recall run below was made after the 07/11 rebuild, on 59 documents / 1,389 chunks / 8.4 MB: same corpus, more of it. Two snapshots, not two truths.)

Two-word query about a specific moment in the diary: the BM25 answer was the right two chunks, ranked first, in the time it takes to blink.

How those numbers were measured (the reviewer question worth asking): same day, same VM, 59 documents and 1,389 chunks in an 8.4 MB index; the lexical side is SQLite FTS5 with the unicode61 tokenizer; median of 5 runs per query with a warm page cache — 0.1 ms for the ranking call and 68 ms