Surfacing Neural Activations Over Multi-Turn Conversation Improves User Calibration to LLM Behavioral Drift

arXiv:2605.15455v2 Announce Type: replace Abstract: Current human-AI interaction (HAI) only surfaces model outputs, hiding how LLM behavior can shift unpredictably toward unsafe behaviors in multi-turn conversations, potentially undermining users' goals or wellbeing. To address this harmful opacity in multi-turn HAI, we introduce an interface that surfaces an LLM's internal activations to help users anticipate and evaluate model behavior. We construct behavioral scores from these activations for six HAI-relevant traits using mechanistic interpretability methods, then visualize them via a dynamic sunburst, drift panel, and attentional cues. In a counterbalanced, randomized controlled study (N=246), both static and dynamic transparency interfaces improved participants' anticipation and evaluation performance (d=-0.36) while helping temper overconfidence in their predictive abilities. Furthermore, the multi-turn interface significantly outperformed the static visualization (d=-0.32) while remaining similarly usable. Our findings suggest that continuously surfacing internal model representations to everyday users is a meaningful and tractable step toward continuously transparent and informed HAI.