A 9B assistant built from our specs went from 6 to 32 right answers out of 32. Most of the fixes were code, not prompts

A personal assistant is easy to demo and hard to trust. The demo ends when the assistant answers. Trust starts when it can change your real calendar, when the model is small enough to run on your own computer, and when it says "done", and you need to know whether that's true.

We built Atlas, a personal assistant for one person. It runs on your own computer: an open-weight model in Ollama, your calendar over CalDAV with an app password, your tasks in a Markdown file, a memory you can read and edit, a page on your phone carried by Tailscale, and a briefing sent to your lock screen at seven by web push. This is an exchange from its phone page, word for word:

You: Move my dentist appointment to Friday at 10.
Atlas: The proposed change is waiting for your yes. Would you like to move the dentist appointment from Thursday 29 October at 16:00 to Friday 30 October at 10:00?
The program: Move "Dentist" from Thursday 29 October at 16:00 to Friday 30 October at 10:00? Yes · No

Here's how we got there, measured.

1. Measure first: 32 requests

We wrote 32 requests against a sample week, each with what a right answer does. Eight read the calendar ("What's on my calendar on Thursday?"), eight change it ("Push my 1:1 with Marcus to 4 pm."), six touch tasks, three memory, three the weather, and four need no tool at all ("Tell me a short joke."). A short program sends each one in a fresh conversation and records every tool the model asks for. A request that changes something counts as right only if the program asks the right question; a request that only reads counts as right only if the model calls the right tool, proposes nothing, and the program didn't have to correct its reply.

The first build, made from our specs before any tool instructions, with qwen3.5:9b: 6 of 32. It called a tool for two task questions and nothing else. It asked which year and month "Thursday" was in. It said there was no brunch on Sunday without looking. And six times it claimed a change it never made:

"I've added a haircut event for Saturday at 2 pm lasting 30 minutes."

Nothing had been added. That's the failure that matters most, because nothing else catches it: there was no question to answer No to.

2. The program asks, not the model

Every tool that changes something comes with a second function that writes the question. When the model asks for move_event, the program doesn't run it. It finds the real event by words from its title, writes the question from what it found, and tells the model that nothing has changed:

def run(self, name: str, args: dict, pending: list[Pending]) -> str: """Run one tool call, or propose it when it would change something.""" if name not in self.tools: return (f"Error: there is no tool called {name}. " "Use only the tools you were given.") try: tool = self.tools[name] if tool.confirm: question = tool.confirm(**args) pending.append(Pending(question, tool, args)) return (f"Proposed: {question} Nothing has changed: I answer Yes or No " "after your reply.") return tool.run(**args) except Exception as error: # reported to the model, which can try again  return f"Error: {error}" 

Because the program writes the question from the real event, the question is right even when the model misunderstood you. The 9B's one remaining miss shows it: asked to push a Friday meeting "to 4 pm", it proposed Monday 26 October at 16:00, and that's exactly what the question on the screen said. You answer No.

One more rule came from a fresh coding session that built this from our specs: the question is asked again before a Yes runs anything. Tick off task 1, and task 3 becomes task 2; a second proposal for "task 3" made before that would tick the wrong one.

3. Wording got half way

The next fix was instructions: for each tool, one line on when to call it, in plain words, written for a small model. And the coming fourteen days, each with its date, so that "Friday" and "next Monday" are lookups:

- The coming days: Monday 26 October (today) = 2026-10-26; Tuesday 27 October (tomorrow) = 2026-10-27; ... Monday 2 November next week = 2026-11-02; ... 

On our own reference code, in early single runs judged a little more loosely, the 9B started at 15 with short instructions. Per-tool instructions took it to 22, and finding events by title words, instead of asking the model for an event ID, plus a first reminder took it to 28.

4. Code got the rest: check every reply

The rest came from code that reads every reply before you see it:

• A claim is corrected, never hidden. "I've moved it" with a question waiting gets "(Nothing has changed yet: answer the question below.)" appended. With nothing proposed: "(Nothing was changed: no change was proposed. Ask again.)" Right after a Yes, the claim is true and passes.
• One reminder of each kind. When a reply announces a tool instead of calling it ("Let me check your calendar."), asks whether to go ahead with a change, talks about the calendar without looking, or gives up before searching the whole coming week, the program sends the model one short message, as if from you, and the model answers again. Each kind at most once per message, so it can't loop.
• Defaults for "optional" arguments. Described as "the day it is on now, or an empty string", on_day was sometimes left out altogether; the call failed and the model asked you for the date. A default in the Python function fixed it.

Final, three runs each: 31 of 32 with qwen3.5:9b, 32 of 32 with qwen3.8:27b, and 27 to 28 with qwen3.5:4b, whose misses are mostly a tool called when none was needed and a due date nobody asked for, each shown or harmless. The build made from our specs alone, measured after the instructions and checks were added: 32 of 32, three times.

5. Small things the measurements found

• The 27B leaks its draft. With thinking off, qwen3.8:27b often wrote a draft, then </think>, then its answer again. Keeping what follows the last </think> removes it.
• Data in the shape the model reads. The briefing's data said "3 other open tasks" when nothing was due; the 9B turned it into three tasks "waiting for you today". A place in brackets, "(Tatte, Copley)", became lunch "with Tatte and Copley" for the 4B. Both are now written so they can't be misread.
• A conversation never outlives its day. The phone page keeps one conversation open for weeks; its "today" would stay Monday. A fresh session noticed. A new day now starts a new conversation.
• A page on your phone needs four checks, not three: a site that renames itself 127.0.0.1 could send your own Tailscale login header from its own page. The header is now accepted only on your computer's ts.net name, which no site can point at your machine.

What we'd tell anyone building one

Measure before you tune. Let the program write every question that changes something. Treat wording as half the job: the half that's left is code that reads what the model said and either corrects it or sends it back once.

This comes from Build Your Own AI Butler, Volume 1 of Weekend AI Projects, a book we publish. The free sampler has the opening pages and the full contents: https://weekendaiprojects.gumroad.com/l/sampler. The book itself is at https://weekendaiprojects.gumroad.com/l/ai-butler.