Facilitating Reproducibility in Visualization Systems Research Through Containerization

Authors

Thomas Breimer (Brandeis University), Pirm Dhararag (Brandeis University), Kiya Gedamu (Brandeis University), Paras Pandey (Brandeis University), Marie Amandine Reyes (Brandeis University), Tal Spector (Brandeis University), Galileo Steinberg (Brandeis University), Nathan Subrahmanian (Brandeis University), Paul Weir (Brandeis University), Peter Williams (Brandeis University), Weichen Zhao (Brandeis University), Dylan Cashman (Brandeis University)

Links

Keywords

Visualization systems, containerization, reproducibility, software architectures, open science

Abstract

In this position paper, we argue for the importance of containerizing visual analytics artifacts to improve the scientific progress of our research community. First, based on our previous experience authoring, reading, and reviewing visual analytics research from the previous decade, we identify risks and impediments to empirical research in visual analytics within the broader reproducibility crisis, including an inability to compare systems and techniques and a slow pace of adjusting to a rapidly changing ecosystem of analysis methods, such as machine learning and artificial intelligence. Next, we analyze the benefits of different artifact sharing and deployment setups across five artifact goals: reproducibility, replicability, portability, performance, and scalability. Then, we argue that containerization addresses the issue of reproducibility in visual analytics research, with downstream benefits in terms of replicability, portability, performance, and scalability. To demonstrate the ease of use and the utility of containerization, we deploy ten visual analytics systems developed over the course of a graduate seminar onto a live server using containerization with the help of agentic AI. We share lessons learned through the containerization process, and describe a suite of potential benefits for our research community as well as research opportunities for leveraging modularity to improve the artifacts themselves.