March 31 has come and gone, which means that the IEEE VIS 2024 full papers deadline has passed. This is when us OPCs and APCs are starting our work, and when the PC and eventually the entire reviewer community will get involved. What is the task? Why, reviewing all those papers that were submitted to the annual deadline, of course! In this brief post, we will give some numbers about the submissions and then outline the next steps in the process.

A total of 680 abstracts were submitted for the abstract deadline on March 21. Out of those, 557 full papers were eventually submitted, a conversion rate of 82%. The number of submissions is up; in 2023, the conference saw 539 submissions out of 635 abstracts (85% conversion), and in 2022, there were 460 out of 560 abstracts (82% conversion). This means submissions increased by 3.53% from last year, and 21% from 2022. This is a fairly small increase compared to the 17% increase from 2022 to 2023. However, the conversion rate of abstract to full paper is consistent.

All APCs pull a very high load and we are grateful for their efforts. Still, some areas received more submissions than others. For the specific areas, Applications (Area 2)—not surprisingly—received the most number of submissions: 154. We wish Tatiana von Landesberger (University of Cologne, Germany) and Jiawan Zhang (Tianjin University) all the best with handling all these submissions! Second came Theoretical & Empirical (Area 1) with 112 submissions. This area is spearheaded by APCs Adam Perer (Carnegie Mellon University) and Matthew Kay (Northwestern University). This is narrowly followed by Representations & Interaction (Area 4) with 110 submissions, and stable hands Daniel Keefe (University of Minnesota) and Pierre Dragicevic (Inria Bordeaux) at the helm. Analytics & Decisions (Area 6) APCs Wenwen Dou (University of North Carolina at Charlotte) and Steffen Koch (University of Stuttgart) are handling 77 submissions. Data Transformations (Area 5) has 53 submissions, and is managed by Filip Sadlo (Heidelberg University) and Ivan Viola (King Abdullah University of Science and Technology). Finally, weighing in at 52 submissions is the Systems & Rendering area (Area 3) with Chaoli Wang (University of Notre Dame) and Christoph Garth (University of Kaiserslautern-Landau) at the wheel. Thanks all to these APCs who make our jobs as OPCs possible!

Supporting these 12 APCs is a program committee of 143 hard-working PC members who can expect to handle somewhere between 6 to 8 papers each (split between the role as primary or secondary reviewer). This PC, incidentally, was recruited earlier this spring from a total of 181 invitations. If you are thinking about serving on the VIS program committee in the future, we hope you look out for our annual PC volunteering deadline sometime next fall! We are deeply grateful to these PC members as well as the external reviewers they will be inviting to review these submissions in the weeks to come.

From this point onwards, the VIS review process will proceed with the PC member assignments, which will be released before April 8. After that point, secondary reviewers will look for an external reviewer for each submission. Remember that this year, there is only one external reviewer assigned to each paper. The initial batch of external review invitations will be released on April 11. The external reviews will be due on May 8. The reviews from primary and secondary PC members are due at the same time. Then, discussion will take place between all reviewers on the paper, and the primaries have until May 15 write their summary review. First-round notifications are expected to be released on June 6.

We OPCs are quite excited about all this and are eager to tackle this hard but important work. During our first skim of the submissions, we have seen some very impressive, creative, and potentially important papers. Thanks for submitting to VIS 2024, and we wish you the very best of outcomes for your hard work!

Read more »

We’re entering March and the game is heating up! With less than three weeks to go for the VIS 2024 full papers deadline, it is time that we OPCs start turning to critical matters. One of those is conflicts of interest (COIs). In the last installment, we talked about the importance of volunteering to review papers for the conferences to which you submit papers. However, it is equally vital not only that you are reviewing papers, but that you don’t review papers for which you have a conflict.

What is a conflict of interest? COIs arise because of relationships you have with an author or their institution that could affect your judgment of the work or the community’s perception of your judgment. Note the last point. You may be a person of exceptional integrity that could objectively handle reviewing even your own sibling’s work fairly and with no special treatment, but reviewing your own sibling’s paper would look bad for everyone else. One of the pillars of scientific peer review is that it is objective, and that this objectivity can be confirmed by everyone. Conflicts arise in many ways and have varying durations; see the IEEE VGTC reviewer ethics guide for all the details. Family relationships are obviously conflicts with no expiration, but those are relatively rare and quite straightforward. Another conflict is the academic equivalent of a family relationship: your Ph.D. and postdoc advisor, or anyone who has had a close mentorship role for you, are “forever conflicts”. The same is true in the other direction, i.e. for your advisees. Your close relationship means that neither of you can be expected to (or perceived to) treat each other objectively. The same can be said about close personal friendships (or, for that matter, personal animosities, which hopefully are rare).

Most other conflicts have a time expiration, and for IEEE and VIS, this expiration is three years. In other words, once a relationship has ended (e.g. collaboration on the same paper), you can consider the conflict gone after three years.

The rule of thumb is the same for conflicts with an expiration as those with none: is there a relationship that would (or would be perceived to) affect your ability to treat a person objectively? Conflicts with an expiration include sharing an affiliation, co-authorship on published work, working on the same research project or grant, or similar. Service commitments are special: you are obviously not conflicted if you serve on the same program committee, because then basically everyone in the community would be conflicted with each other. In general, even if you work with somebody closely on a small committee over an extended period of time, conflicts do not automatically arise; however, if you become sufficiently close to somebody that it feels like a conflict, then do declare it. Use your best judgment here.

As an aside, us OPCs are selected not to be conflicted with each other because we need to be able to handle each other’s submissions and conflicted submissions.

If you have a conflict with a paper, you should not be involved in any formal publication decisions regarding it. For external reviewers, this means not reviewing papers you are in conflict with. For PC members, this also means not reviewing such papers, and informing the papers chairs immediately if you are assigned to such a paper. For APCs at VIS, this means that if both APCs are in conflict, the conflicted paper will be moved to a different area to avoid the conflict. If just one APC is in conflict, one of the OPCs will step into their place. For OPCs, there is no such option, so in these situations, the conflicted chair will have to recuse themselves from any decisions involving the conflicted paper. Practically speaking, this will mean not participating in the Zoom call when discussing it. The PCS submission system provides good support for handling conflicts, by ensuring that papers chairs cannot see information about such papers.

Here’s the final question we want to cover: how do we detect these conflicts in the first place? This is where you come in. If you are a reviewer for VIS 2024 (either as a PC member or as an external reviewer), it is your responsibility to ensure that your affiliation is correct and that your conflicts have been updated in PCS (the submission system). Because VIS allows for double-blind submissions, we need PCS to flag situations when there is a conflict even if you as a reviewer don’t see the author names. Even for single-blind submissions, where the author names are visible to you, correct affiliations and declared conflicts will minimize situations where you get assigned a paper you really shouldn’t review, and then we all have to go through the hassle of getting the paper reassigned to another reviewer.

Many early-career researchers in the community will have recently changed affiliations, for example Ph.D. students who have graduated and moved on to new institutions. Please make sure you have updated PCS with your new primary affiliation, and also keep your old institution listed as your secondary affiliation for 3 years—the three-year rule applies here as well. Conversely, once those 3 years are up, please remove that institution from your own affiliation list. If you are still actively collaborating with people there, that should be handled through the usual conflicts identification mechanisms in PCS, not through your affiliation.

Speaking of PCS and conflicts: the good news is that PCS will now help accelerate the process of declaring your conflicts by checking against all recent submissions in its database. The bad news is that it’s sometimes over-enthusiastic: it uses a 4-year window rather than a 3-year window, and includes a few tracks where we do not consider conflicts to occur (such as shared participation in a panel or tutorial). So do take a close look at the automatic inferrals, in addition to entering information about any other conflicts such as new collaborations where you have not yet submitted papers together as co-authors.

With all this information fresh in your mind, please take a moment to go into PCS and make sure your affiliation is up to date. (If you’re on the VIS PC, you will also be asked to update your conflicts after the abstracts submission deadline.) Providing all this information already now will make everybody’s life easier.

Read more »

In the previous post by the The VIS Executive Committee (VEC), we outlined our initiative to investigate the wicked problem of planning future VIS conferences, focusing on the extent to which the conference should assume a hybrid format or consider other alternative formats to broaden participation, while ensuring positive experiences for both in-person and remote attendees.

We now continue this initiative with an overview of VIS attendance patterns between 2018 and 2023. This period is particularly interesting given the variation in location / time zone and format of VIS during this time:

• 2018: Berlin (in person, GMT+1)
• 2019: Vancouver (in person, GMT-8)
• 2020: Salt Lake City (virtual, in GMT-7)
• 2021: New Orleans (virtual, in GMT-6, with several in-person satellite events around the world)
• 2022: Oklahoma City (synchronous hybrid, GMT-6)
• 2023: Melbourne (in person, GMT+11)

For each year listed above, we collected anonymized registration data aggregated by world region and country. For the synchronous hybrid conference of 2022, we also aggregated the registrations by registration format (in-person or virtual). We acknowledge that when an attendee registers for VIS, their response to the location fields in the registration form may not necessarily represent their country of origin, but rather where they are currently employed or studying. Furthermore, there is no way of verifying if the self-reported country for a person who registered to attend VIS virtually between 2020 and 2022 was the country that they were physically in during the conference. Lastly, we do not currently have any data regarding the level of engagement for virtual attendees beyond registration.

Caveats aside, we report on attendance for the benefit of those considering to make a bid to host VIS in the coming years, VIS organizing committee members planning and scheduling virtual or hybrid program content, as well as for those inclined to host satellite events, such as the eight events that coincided with VIS 2021 or the asynchronous VIS 2023 satellite event in France.

Overall VIS Registration Numbers

VIS 2018 in Berlin remains to be largest VIS conference to date in terms of in-person attendee count (over 1,200), while VIS 2020 had the highest number of virtual registrations (nearly 6,000). We also saw nearly 3,700 virtual registrations in 2021 and nearly 700 in 2022 (the latter also had over 600 in-person attendees). Most recently, VIS 2023 attracted over 800 in-person attendees.

VIS Attendance by Region

Irrespective of the means of attendance (in-person / virtual), we saw some notable differences in terms of the proportion of attendees from different world regions (◼︎ = Europe; ◼︎ = Americas; ◼︎ = Asia; ◼︎ = Oceania; ◼︎ = Africa).

When VIS was in Germany (2018), a little over half of the attendees were based in Europe and made the short trip to Berlin; about a third flew in from North America, just over 10% flew in from Asia, and 1% traveled from somewhere else. The next year (Vancouver), the relative proportions of Europeans and North Americans inverted. Notably, over the next three years, the relative proportions of attendees from North America, Europe, and Asia remained somewhat stable. While the VIS program was virtual in 2020 - 2021 and hybrid in 2022, recall that each program was scheduled in time zone that was convenient for those in North America. Finally, at the most recent VIS conference in Melbourne, which had no virtual attendance option, we saw a fairly even split in terms of who made the trip to Australia: about 30% flew in from North America, 26% came from Asia, 24% were based in Oceania, and 19% made the long trip from Europe.

Virtual VIS Attendance

A couple of questions arise from our virtual registration numbers: (1) Did VIS have a broader geographic reach during the virtual-only years (2020-21)?; and (2) Who opted to attend the hybrid VIS 2022 conference virtually?

Among the virtual registrations, the majority were from those based in world regions and countries already well-represented at VIS: North America, Western and Northern Europe, and East Asia. However, we made special note of the countries that were not represented among in-person registrations between 2018 and 2023. Countries in Southern and Southeast Asia stand out in this pack, with a total of over 100 registrations between 2020 and 2022. About half as many registrations over this period were from those based in Eastern and Southern Europe. Finally, we saw a relatively small number of registrations from Africa and the Americas outside of North America.

Looking at VIS 2022 in particular, a little over 40% of the nearly 700 virtual registrations were from those based in the Americas, while a little under 30% were based in Asia, and just over a quarter were based in Europe. Across all of these registrations, only a couple dozen (~3%) were from those based in countries that had no in-person representation across the years surveyed.

Implications

Aside from the global COVID-19 pandemic that prevented international in-person gatherings in 2020 and 2021, the registration data by itself doesn’t tell us why participants opted to attend virtually in 2022. Nor does it tell us which factors matter to prospective VIS attendees mulling over whether to attend in person or virtually; this is the focus of our the survey announced in our earlier post, and we will present the results of this survey in a future post. However, the registration data does suggest that there is not a large difference in terms of global representation between in-person and virtual attendees.

While attending a regional satellite event may not be accessible or appealing to those who decide against traveling in person to the main event, the virtual registration numbers suggest that there may be a large enough contingent of people in Europe and East Asia to merit satellite events in those locales when VIS takes place in North America. There may even be a critical mass of people in South Asia and Southeast Asia willing to attend such a regional event.

While just over 40% of the virtual registrations for VIS 2022 were from those based in the Americas, the synchronous virtual conference experience for the remaining ~60% was likely to occur at an inconvenient time of day. To accommodate virtual attendees of future VIS conferences, organizers should consider the benefits and trade-offs of asynchronous or partially-synchronous virtual attendee experiences. For example, let’s imagine a future VIS conference hosted somewhere in North America. Presentations delivered by in-person attendees could be recorded during the day and uploaded in the evening, at which point they could be viewed asynchronously by remote attendees. Early the following morning, those whose talks were recorded could join a moderated virtual Q&A session with remote attendees in Europe and Africa, and later that day, they could join a second moderated virtual Q&A session with remote attendees in Asia and Oceania.

At VIS 2024 in Florida, the general chairs have already determined that paper presentation sessions will be split between those delivered by in-person attendees and those delivered by remote virtual attendees. Should this split continue in future VIS conferences, the virtual registration demographics suggest that these virtual-only sessions be scheduled in the early morning and late afternoon / early evening, so as to respect the typical working hours of those attending from around the world.

Read more »

In this edition of the Road to VIS 2024, we want to tackle the topic of reviewing and reviewer fatigue. We want to shed light on the invisible labor that our hard-working reviewers do for every paper submitted to the conference. Our goal is to enlighten and encourage, not to reproach. But yes, if this post motivates you to go to PCS right now and volunteer to review papers for VIS 2024, so much the better. (We mean it—do it now!)

The truth of the matter is that every paper you submit to IEEE VIS comes at a cost: the unpaid labor of at least three (four in earlier years) reviewers who have volunteered to evaluate your work. In addition, work is done by the Area Papers Chairs (APCs) who spend time finding suitable primary and secondary reviewers to handle your submission, and then tries to make a decision about your paper based on the reviews, not to mention us Overall Paper Chairs (OPCs) who facilitate this accept/reject decision-making process at a global level. Taken together, there are probably dozens of hours invested in each paper you submit.

This is all to say that while it may seem like submitting a paper is a little like playing the lottery for free—after all, you don’t have to pay anything to enter—there is a very real cost in human labor for every paper submitted to the conference. At the same time, many of us may admit to at one point or another saying something along the lines of “let’s submit this paper even if it’s not entirely ready—at least we’ll get good feedback.” Yes, there is often good feedback for IEEE VIS—our reviews tend to be of remarkably high quality—but the primary goal is to select high-quality papers with the potential for being accepted and nurture them to that point. An additional outcome of the review process is to provide good feedback to authors on the strengths and weaknesses of the work that is rejected, but you should not treat the IEEE VIS review process as a free editing or research critique service. Of course, it’s difficult to predict which papers will shine and which will be deemed premature—many of us have been surprised by our favorites being rejected and our long-shots accepted, so we don’t wish to discourage you from submitting work that you personally consider above the bar for publication! Rather, this post is intended to raise awareness that if we treat all of these volunteer efforts as unpaid labor that we can leverage for our own good, the system may not be sustainable.

Since reviewing is not free, it is important that authors devote time to it to keep the scales balanced. Given that three reviewers handle each paper you submit, you and your co-authors should plan to review at least three other papers in return (split between all co-authors). If you do not review your share, you are essentially benefiting from other people’s labor without contributing your own. This approach is not equitable, and in the long run it could lead to the collapse of the peer review system.

Some conferences are already seeing indications that the system is beginning to fail, particularly since the one-two punch that started with the pandemic, and continues to this day: most venues experienced a significant increase in submissions, even as more review requests were declined. VIS has not been spared—in past years, we heard tales of PC members having to ask half a dozen or more potential external reviewers before someone finally accepted the request to review a specific paper. Reviewer fatigue is real, and the only antidote is that every author pulls their own weight. ACM CHI, for example, now requires that all submissions list at least three co-authors who commit to review papers in return. With VIS, we are taking the slightly more gentle approach of strongly encouraging this level of commitment, although not yet formally requiring it.

As always in life, there are exceptions to the hard and fast rule of reviewing at least as many papers as your own paper receives. For example, junior students are often exempted from reviewing, mainly because they don’t possess the necessary experience and judgment to take on this task. Even senior authors may be excused if they have significant service commitments elsewhere. For example, none of the OPCs and APCs this year will be reviewing VIS papers because our time is already committed. Still, our philosophy is that if you take the time to author research for a conference, you should also make time to review for that conference. Please keep this principle in mind as external review requests start to appear in early April.

Read more »

With deadlines quickly approaching, you may be curious about the format of IEEE VIS 2024. VIS 2024 will be a hybrid event, but as we all know, that can mean many different things. As we considered possible formats, we tried to balance the desire to maximize the experience for in-person participants while weighing the importance of connecting to remote participants in our community. We also considered the cost, both dollars and work hours (we are all volunteers, after all), associated with the additional needs of a hybrid conference. Finally, we wanted to test a model that could be sustainable for future VIS conferences, which have different space and financial constraints than our own.

Our determination was that different aspects of the conference had differing needs: some portions would be zoom-hybrid, some would be streamed, some would be in-person only, and finally, some would be primarily virtual.

Full and short papers: Full and short papers will be split into in-person presenter-only and virtual presenter-only sessions.

• In-person presenter-only sessions will be streamed, potentially with a time-delay. Q&A will be limited to in-person attendees.
• Virtual presenter-only sessions will be hosted on Zoom, with Q&A available to all participants (remote and in-person). In addition to being streamed, each virtual session that occurs during normal conference hours will be played back in a room at the in-person conference.

» Authors of full and short papers will be required to make a commitment to the in-person or virtual track during their second-round submissions. «

Workshops, panels, and other associated events: Workshops, panels, and other associated events will be equipped with Zoom-hybrid capabilities at the conference. However, it will be up to the discretion of the organizers of each event to determine how these capabilities will be utilized.

Plenary Sessions: All plenary sessions, including the keynote, capstone, best papers, etc., will be streamed this year, potentially with a time-delay. As such, Q&A will be limited to in-person participants.

Posters, VISAP, meetups, etc.: The conference will support these events in-person only.

Finally, a note about registration—Each full and short paper accepted at IEEE VIS 2024 will be required to be presented by one of the authors, either in-person or virtually. All presenters are required to register as a speaker, irrespective of their mode of presentation. Additionally, diversity and inclusivity scholarships will be available to support the participation of speakers with financial needs.

There remain many details to be worked out, and we will continue to communicate those details as they become available. The general chairs remain open to your feedback and questions. We look forward to seeing you, in-person or virtually, at IEEE VIS 2024!

IEEE VIS 2024 General Chairs
Paul Rosen
Kristi Potter
Remco Chang

Read more »