January 2017
Josh Tenenberg, Donald Chinn, Judy Sheard, Lauri Malmi
The program for ICER 2016 and several years prior was determined by a five-person program-selection committee. This program-selection committee was composed of the conference chairs for the current year, the previous year, and the subsequent year’s ICER conference, plus two invited associate chairs. Each submitted paper, which was assigned a minimum of three reviewers, was also assigned to two program-selection committee members, who served as meta-reviewers for this paper. The program-selection committee made program acceptance determinations through a series of electronically-mediated conference calls during which each of the paper submissions was discussed. For more details, see the page on Review Process for ICER 2016.
A number of problems emerged:
- With over 100 submissions for 2016, each program-selection committee member was required to carefully read over 40 papers and their reviews, as well any additional reviewer discussion that might have occurred (anonymously, mediated through EasyChair). Much of this activity occurred during a two-week period.
- Given the rotation of conference chairs across continents from one year to the next, conference calls occurred across three continents, with at least one person each from Australasia, Europe, and the United States. This meant that at least one (and often 2 or 3) of these people were meeting during a time that was normally when they slept.
- Conference chairs participating in this process did so for three consecutive years. This was particularly onerous in the middle year, when the conference chair was also significantly involved in the local organization at the conference site.
As a result, the 2017 conference co-chairs determined that this review process was unsustainable, particular as the conference continues to grow in size. In addition, given the high meta-reviewing and decision-making burden across time zones with very tight time constraints, the conference co-chairs felt that this process could no longer be relied upon to provide defensible decisions.
The primary changes made to this review process starting with ICER 2017 are:
- The program-selection committee is reduced from 5 members to 2 members, and renamed the Program co-chairs.
- In prior years, appointment as conference chair required 3 years of service in the program-selection committee. Under the new process, program co-chairs are different organizational roles than local organizers of the ICER conference. Therefore program co-chairs can be appointed based on their research qualifications and not their local organizational skills or the suitability of their home institution to host ICER. In addition, service as program co-chair is now a 2 year (not a 3 year) commitment.
- What used to be called the Program Committee is renamed the Review Committee. This will have about the same number of members as in the past.
- A new Program Committee is created, with each member responsible for writing meta-reviews. A limit of three years is placed on the term of service, though reappointment is allowed after two years of not serving on the Program Committee. The expected number of members is 15-20.
- Two additional weeks are added to the review process to allow sufficient time for the PC members to write meta-reviews.