Visual Decoding Operators: Towards a Compositional Theory of Visualization Perception
Authors
Sheng Long (Northwestern University), Remco Chang (Tufts University), eugene Wu (Columbia University), Alex Kale (University of Chicago), Matthew Kay (Northwestern University)
Presentation
- Session
- Did you see that? Are you sure?
- Time
- Wednesday, Nov 11, 13:00 – 13:12 (US/Eastern) · session 13:00 – 14:30
- Location
- Hall Essex center
Links
Sign in to access the preprint PDF.
Sign in- Download Supplemental Material
Keywords
Visualization Theory, Visual Decoding Operator, Composable Models, Sensor Fusion, Perceptual Effectiveness
Abstract
Prior work on perceptual effectiveness has decomposed visualizations into smaller common units (e.g., channels such as angle, position, and length) to establish rankings. While useful, these decompositions lack the computational structure to predict performance for new visualization × task combinations, requiring new experiments for each. We propose an alternative unit of analysis: operationalizing quantitative visualization interpretation as sequences of composable visual decoding operators. Using probability density function (PDF) and cumulative distribution function (CDF) charts, we examine how four chart-specific tasks can be decomposed into five reusable, chart-agnostic perceptual operations and characterize their error profiles through hierarchical Bayesian modeling. We then test generalizability by composing one kind of learned operators to predict performance on a structurally different task: Moritz et al.’s [37] scatterplot mean-estimation experiment, where the chart type, chart dimensions, and analytic goal all differ from the learning conditions. With a pre-registered analysis plan, we compose operators under six candidate strategies and evaluate each against empirical data with no parameters fit to the response data. One strategy captures both bias and variance of observed responses; five alternatives fail in distinguishable ways. We argue that this decoding-operator-oriented approach to empirical visualization research demonstrates the feasibility of a different way of doing empirical visualization research, one where findings compose, and predictions extend beyond the conditions in which they were measured. Free copy of this paper and supplemental materials: https://osf.io/prtfq.
For Practitioners
This paper is most directly useful to visualization and UX researchers who run perceptual studies, and to toolsmiths building recommendation or authoring systems. Researchers can reuse our task-decomposition methodology, open reVISit implementation, and visual-angle calibration procedure to produce findings that compose across studies rather than remain bound to a single chart × task pairing; toolsmiths can use operator-level bias and variance as a computable proxy for expected reading error, ranking candidate designs without commissioning a new experiment for each combination. For practitioners who design charts of distributions and uncertainty, the takeaway is where error enters: reading a median off a CDF is not one judgment but two chained projections, so interventions that shorten a projection (reference lines, direct annotation, anchoring gridlines) target the actual error source rather than the chart as a whole. Because operators are parameterized in visual angle, this reasoning transfers across phone, laptop, and large-display contexts.