Explainable Mapper: Charting LLM Embedding Spaces Using Perturbation-Based Exploration, Explanation, and Verification Agents

Authors

Xinyuan Yan (Scientific Computing and Imaging Institute), Rita Sevastjanova (ETH Zurich), Sinie van der Ben (ETH Zurich), Mennatallah El-Assady (ETH Zürich), Bei Wang (University of Utah)

Presentation

Session

Talk to my agent

Time

Tuesday, Nov 10, 10:24 – 10:36 (US/Eastern) · session 10:00 – 11:30

Location

Hall Essex center

Keywords

Topological data analysis, explainable AI, large language models, visual analytics, embedding spaces

Abstract

Large language models (LLMs) produce high-dimensional embeddings that encode rich linguistic structure. We leverage mapper graphs, a technique from topological data analysis, to examine the organization of these embedding spaces, where nodes represent clusters of embeddings and edges capture their overlap. Although mapper graphs can reveal meaningful patterns, interpreting the linguistic properties they encode remains time-consuming and labor-intensive. We introduce Explainable Mapper, a framework for semi-automatic annotation and systematic exploration of mapper graphs that enables fine-grained analysis of embedding spaces. Explainable Mapper defines a taxonomy of explorable elements, including nodes, edges, paths, components, and trajectories, and integrates two types of LLM-based agents: an Explanation Agent that generates candidate interpretations of mapper elements, and a Verification Agent that evaluates their robustness using perturbation-based strategies. Together, these agents enable semi-automated and stable interpretation of topological structures. We instantiate Explainable Mapper within an interactive visual analytics workspace that combines agent-driven analysis with user-guided exploration, supporting diverse human--AI workflows for scalable and interpretable investigation. We evaluate our approach through case studies on embeddings from multiple transformer models and datasets, as well as an expert user study. Results show that Explainable Mapper reproduces established linguistic patterns while uncovering novel insights into LLM embedding spaces. This work extends mapper-based embedding analysis by integrating LLM-assisted explanation generation with perturbation-based verification, enabling scalable and systematic interpretation of high-dimensional embedding spaces.

For Practitioners

NLP and machine-learning practitioners, computational linguists, explainable-AI researchers, and visual analytics specialists would be interested in this paper. They can use Explainable Mapper as a topological visual analytics tool to explore the structure of language-model embedding spaces through word embeddings, while using its explanation and verification agents to interpret mapper graph structures and assess the stability of discovered linguistic patterns.