What a host-side layer for LLM agents is, in plain words (and what it is not)

2026-11-16来源 dev.toEnglish

This article was created with the help of AI. The AXI architecture is by Rajnish Choubey. The tests were built and run by Shalendra Gaur.

Most talk about agent cost and reliability is about the model. This is about the other side of the loop: the code that holds the state the model is editing and decides what happens to each reply.

The usual loop

An agent edits some state: a plan, a list, a graph, a queue of tickets. In the usual loop the model sends a reply, the reply is applied as written, and the whole history is sent back to the model on the next step. Two things follow. A wrong reply that nothing catches becomes part of the state, and every later step builds on it. And the prompt grows with every step, so each step costs more than the last.

What AXI is

AXI is a layer on the host side, between a model and the state it edits. The host is whoever runs the model and holds the state. In AXI:

  1. The model proposes one change.
  2. The host checks the change against fixed rules.
  3. A rule-based repair completes what the rules can complete. A deleted service, for example, takes its connections with it.
  4. The change is committed all or not at all.
  5. The model is shown the exact state again, in a short prompt built from that state, not from the history.

The architecture is Rajnish Choubey's.

What it is not

  • It is not a model, and it needs no training or fine-tuning. The model is unchanged.
  • It is not a prompt trick and not a memory for chat. The state is held by the host and checked by code.
  • It does not make a model reason better about things outside the state the host holds. The claims are about the host-held state.
  • The repository is a reference prototype with tests and saved results. The deeper work of the architecture is Rajnish Choubey's and is not in it.

Why the host

The host is the only place that knows the true state and can enforce rules on it. The model can be told a rule and often follows it, but the host can check every reply whether or not it does. That also lets the prompt be rebuilt from the state at each step, so it does not grow with the length of the job. It works from the first step.

What has been measured

On one model (Gemini 3.8 Flash) and one job (editing a service graph), over 4,000 requests, the host-held state was exactly right on 4,000 of 4,000 steps. The model's own replies were exactly the ideal reply 3,954 times in 4,000; a fixed rule completed the 46 others. On the first 400 requests, applying the same replies as written left the state exactly right on 85 steps.

What this does not show

It is one model, one job and one sampled run, and the repair rule is the job's own rule. How well it carries to other jobs is the open question, and it needs other jobs to answer it. If you run models and agent loops and want to try it on your own workload, the tests and a notebook that uses your own key with a spend limit are in the repository.

Repository: https://github.com/gaurshalendra2412/axi-runtime

White paper, "AXI: The Cure for the Cost Crisis in AI" (Rajnish Choubey and Shalendra Gaur): https://doi.org/10.5281/zenodo.23958737

原文:https://www.a8q9.cn/en/article/what-a-host-side-layer-for-llm-agents-is-in-plain-words-and-what-it-is-not-bcf61a.html

相关文章