Language models generate maximally activating examples for any residual direction

Researchers develop Maximally-Activating Example Metamodels (MAEMs), language models that map a residual-stream direction from a subject model to text exemplars strongly activating it. A label-free reinforcement-learning objective maximises cosine similarity between the supplied direction and activations produced by MAEM-generated text, avoiding corpus limits.

Why it matters

Corpus scanning cannot reveal activations for rare directions; generating examples could make interpretability cover the full residual stream.

Analysis

Interpretability may move from searching existing text to generating targeted probes for model internals.

Source: arXiv

AI disclosure: this article was produced with AI assistance and reviewed for accuracy.