MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction
Authors
Yue Zhao (Shandong Second Medical University), Hongxu Liu (Shandong University), Feiyu Wang (Shandong University), Xiaoyu Yang (Shandong University), Tong Ge (Bairong Inc.), Zhen Yang (Shandong University), Chao Wang (Ke Holdings Inc.), Qiong Zeng (Shandong University)
Presentation
- Session
- Let's dig into the data (from China)
- Time
- Friday, Nov 13, 08:36 – 08:48 (US/Eastern) · session 08:00 – 09:30
- Location
- Hall America center
Links
Sign in to access the preprint PDF.
Sign in- Download Supplemental Material
Keywords
Multimodal LLM, Coordinated Multi-View Interface, Benchmarking, Interaction and Data Binding
Abstract
Multimodal large language models (MLLMs) are increasingly expected to automate visualization development by generating code directly from visual designs. However, existing evaluations focus on single-chart generation, overlooking coordinated multi-view interface construction, a practically demanding task that requires jointly managing data semantics, multi-view coordination, and interactive logic. As a result, MLLMs' capabilities in this area remain largely unexamined, while the field also lacks a dedicated benchmark for systematic assessment. To address this gap, we introduce MV-Bench, the first benchmark for evaluating MLLMs on coordinated multi-view interface construction. Rather than relying on incomplete or inconsistent open-source implementations, we leverage Tableau workbook files as a principled source of ground-truth, as they explicitly encode data bindings, visual mappings, and interactions. We develop a multi-stage pipeline that converts these specifications into executable web-based interfaces via structured intermediate representations. Using this pipeline, we construct 92 base interfaces and systematically recombine them across chart types, datasets, and interaction patterns, producing 1,048 verified benchmark instances with executable code, rendered interfaces, datasets, and interaction annotations. Using MV-Bench, we evaluate five state-of-the-art MLLMs, queried under a single-pass setting, across metrics measuring visual fidelity, data binding correctness, and interaction completeness. While the strongest model achieves 75.45% accuracy in visual layout reproduction, performance drops sharply to 21.71% on data binding and 11.68% on interaction completeness. These results demonstrate that current MLLMs can reproduce visual appearance but remain fundamentally limited in generating the data semantics and interactive logic required by coordinated multi-view interfaces. Further experiments with iterative refinement improve code executability but do not substantially narrow the gap in data binding and interaction generation.
For Practitioners
Visualization developers, BI engineers, data scientists, and builders of AI-assisted coding tools would be most interested. They can use the paper’s benchmark and findings to test AI-generated dashboards for correct data bindings and cross-view interactions instead of judging them only by visual similarity.