Ask HN: Beyond traces, how do you know your agent did the job?

I keep going back and forth on this.

Traces and logs tell me what the agent did: every tool call, retry, token, and step.

That's useful when something breaks. What I'm less sure about is whether that tells me the agent actually did the right thing.

A run can look clean and still miss what I wanted. Or it can "finish" without the outcome really being done. So finished and succeeded aren't the same thing.

I haven't made up my mind how to check for that, but I'm experimenting a lot with extracting intents from my agent and correlating among them.

Curious how you do it:

- Do you capture intent and outcome explicitly (e.g. giving a tool to the agent to "declare" it), or infer them after the fact from the traces?

- What kind of "questions" do you "ask" your traces?

---

Comments URL: https://news.ycombinator.com/item?id=50153451

Points: 1

# Comments: 1