Saying Uncertain, Showing Certain: Evidence-Action Mismatches in LLM Visualization Agents

Abstract

Data scientists are beginning to analyze data alongside visualization agents that both explain their reasoning and produce charts. These two channels can disagree: an agent may acknowledge uncertain evidence in its rationale while emitting a plan whose visual semantics assert certainty. We measure this divergence, without any human evaluation, in rendering-free structured plans from 72 neutral source cases and 24 matched pressure variants, yielding 672 valid outputs from two fixed model case studies, gpt-4o and gpt-5.4-mini. Both models usually acknowledged uncertain evidence yet often did not act on it: this acknowledgment–action mismatch occurred in 41% and 59% of valid uncertain baseline cases, and every action failure violated a prespecified action-consistency rule. Within-call reflection helped one model but not the other. Restricting the action menu removed forbidden action types, yet one model rebuilt the same semantics through allowed parameters. Under explicit pressure to appear decisive, even acknowledgment eroded. For human–agent analysis, the plan, not the rationale, carries the visual claim and is the artifact to check.