Attention-Guided Saliency Maps for Interpreting Visualization Literacy in VLMs

Authors

Maeve Hutchinson (City, University of London), Abderrahmane Wassim Mehdaoui (City, University of London), Pranava Madhyastha (City, University of London)

Presentation

Session: Is the Model Even Thinking?
Time: Thursday, Nov 12, 09:03 – 09:12 (US/Eastern) · session 08:00 – 09:30
Location: Hall America south

Keywords

Vision-language models, Saliency, Explainable AI

Abstract

Understanding how vision-language models (VLMs) interpret data visualizations remains an open problem, and is increasingly important as these models are used for analytical tasks where reliable reasoning is essential. We introduce a lightweight, diagnostic saliency map method tailored for text generation over images using transformer models, the current state-of-the-art models in visualization interpretation. Our approach aggregates the language model's attention over the visual tokens across all heads and layers, then maps this attention back onto the vision encoder's patch grid to localise it over the image, producing a direct correspondence between each generated answer token and the image regions it attended to. This yields fast, gradient-free saliency maps that expose how VLMs allocate focus across visual elements during answer generation, enabling inspection of whether model attention aligns with semantically relevant components. We evaluate our approach using a deletion metrics which validates the causal faithfulness of our saliency maps to the model’s behavior.

For Practitioners

Developers of VLM-based chart-QA and analytics tools, plus data scientists and analysts who rely on these models to read charts, and the visualization-literacy and explainable-AI research communities. They can use the method as a lightweight, gradient-free check that a model's answer was actually grounded in the right axis label or mark, rather than a memorised shortcut, helping to debug failures, audit models for shortcut behaviour, and treat correct grounding as a quality metric alongside accuracy.