Research Papers

Research papers submitted to ICER for peer review are limited to 10 pages plus up to 2 pages for references. Read on for details about preparing your submission, for the ICER research paper review process, and for details about final publication.

Preparing to Submit

Submission Deadlines

Friday, April 3, 2020 Anywhere on Earth (UTC-12) – Mandatory abstract/intent to submit uploaded to EasyChair

All authors intending to submit a research paper must first enter their author, affiliation, title, and text abstract information into EasyChair by this deadline. Your abstract will be used by potential reviewers to bid on papers that match their expertise. Ensure the best reviewer match by writing a clear abstract that describes your paper’s motivation, its theoretical grounding, and the research methods used along with a summary of your results. Authors failing to submit an abstract and associated meta-data by the due date (even if the EasyChair system accepts the abstract late) will not be eligible to upload full paper text.

Friday, April 10, 2020 Anywhere on Earth (UTC-12) – Full paper submission (anonymized) added to EasyChair

Authors have a week following the initial abstract deadline to complete final submission of the paper to be reviewed. This will be submitted as an update to the paper entry created during abstract submission. Please reduce file sizes when possible for ease of downloading, as long as this does not sacrifice the quality necessary for a fair review.

Friday, June 12, 2020 – Author notification

Sunday, June 28, 2020 – Final camera ready submission for accepted papers

Paper Formatting and Length Requirements

Papers submitted to ICER must conform to the style rules outlined on the ACM SIG Proceedings website.

  • LaTeX: There is a single ACM master template; use the latest version posted on the ACM site. Then, use the “ACM Standard” proceedings document class, which is intended for conference proceedings. The master template uses the identifier “sigconf” for this. Your document class declaration at the beginning of your document should therefore be: \documentclass[sigconf]{acmart}.  The easiest way to ensure that you have up-to-date versions and the correct fonts is to use Overleaf, but any reasonable LaTeX environment should suffice if you load the correct templates and install any needed packages.
  • Word:  if you are using Word, use the Interim template at the link above. Do not use the “New workflow for ACM publications”; it’s still being evaluated.

We strongly suggest that you use LaTeX for the time savings in formatting and formatting consistency, but if you must use Word, do your best to ensure consistent formatting. Reviewers will be instructed to ignore minor differences in formatting during review. Accepted papers will need to confirm exactly to the ACM Proceedings format.

Preparing for Anonymous Review

ICER paper submissions are reviewed using an anonymous process, where the authors do not know the identify of the reviews, and the reviewers do not know the identity of the authors. To ensure this:

  • Remove author names. Do not simply cover the details with a black box, as the text can easily be extracted from under the box by dragging the cursor over it. Remove affiliations.
  • Remove any identifying information in acknowledgements.
  • Avoid references to your affiliation, such as your college, university, or company. For example, rather than referring to your actual university, you might write “A Large Metropolitan University (ALMU)” rather than “Auckland University of Technology (AUT)”.
  • Avoid referring to yourself when citing your own work.
  • Avoid titles that indicate a clearly identifiable research project.
  • Omit author details from the PDF you generate, such as author name or the name of the source document.

The anonymization requirements hold for both abstracts and full papers.

Review process

The ICER research paper review process is designed to attempt to maximize the alignment between a paper and the expertise required to review it, minimize unconstructive critique, and completely avoid conflicts of interests.

New rules around conflict of interest are being trialled for ICER2020, as described below.  The changes may or may not be adopted for subsequent conferences.

Roles

There are three roles in the review process:

  • Program chairs. Each year there are senior and junior program chairs. Junior program chairs participate in this year and the following year’s process. Two program chairs are appointed by the SIGCSE board to serve a two year term, based on nominations from the current conference co-chairs. One new appointment is made each year so that in any given year there is always a continuing program chair from the prior year and a new program chair. Appointment criteria include prior attendance and publication at ICER, past service on the ICER Program Committee, research excellence in Computing Education, collaborative and organizational skills to share oversight of the program selection process. The ICER Steering Committee solicits and selects candidates for future program chairs.
  • Meta-reviewers review the reviews authored by reviewers, ensuring that the review content constructive and aligned with the review criteria, as well as summarizing reviews and making recommendations for a paper’s acceptance and rejection. They also provide feedback on reviews if necessary, asking reviews to improve the quality of reviews. The program chairs invite and appoint meta-reviewers. The committee will be sized so that each meta-reviewer will handle 8-10 paper submissions. The program chairs are responsible for inviting new members of the committee and retiring longer serving members, balancing the need for intellectual diversity and expertise necessary for the pool of submissions. Program chairs will prioritize length of service on Review Committee, consistently high performance as reviewer, and history of prior publication in ICER or other computing education research venues. Together, the program chairs and  meta-reviewers constitute the Program Committee (PC).
  • Reviewers write reviews of submissions, evaluating them against the review criteria. The program chairs also invite and appoint the reviewers. The committee will be sized so that each reviewer will serve for 4-6 paper submissions. Each reviewer will serve a one year term, with no limits on reappointment. Appointment criteria include expertise in relevant areas of computing education research and past reviewing experience in computing education research venues. Together, all reviewers constitute the review committee (RC).

Anyone interested in joining the RC or PC can write to the current program chairs expressing their interest.

Conflicts of interest

ICER takes conflicts of interest, both real and perceived, quite seriously. The conference adheres to the ACM conflict of interest policy as well as the SIGCSE conflict of interest policy (with the exception of ICER 2020, in which there will be a trial of allowing program chairs to submit). A paper submitted to the ICER conference is a conflict of interest for an individual if at least one of the following is true: 1) the individual is a co-author of the paper, 2) a student of the individual is a co-author of the paper, or 3) the individual identifies the paper as a conflict of interest, i.e. that the individual does not believe that he or she can provide an impartial evaluation of the paper. No reviewer, meta-reviewer, or program chair with a conflict of interest in the paper will be included in any evaluation, discussion, or decision about the paper. It is the responsibility of the reviewers, meta-reviewers, and program chairs to declare their conflicts of interest. It is the responsibility of the program chairs to ensure that no member of the Program or Review Committees is assigned a role in the review process for any paper for which he or she has a conflict of interest.

Review criteria

Reviewers comment on and score each paper against several criteria on a strongly disagree to strongly agree scale (-2 to 2):

  • The submission is grounded in relevant prior work.
  • The submission leverages available theory when appropriate.
  • The submission describes its methods and/or innovations sufficiently for others to understand how data was obtained, analyzed, and interpreted, or how the innovation works.
  • The submission’s methods and/or innovations soundly address its research questions.
  • The submission advances knowledge of computing education that is of interest to the computing education community.
  • Discussion of results clearly summarizes the submission’s contributions and implications.
  • The submission is written clearly enough to publish.
  • Based on the criteria above, this paper should be published at ICER.

Reviewers are trained on these criteria before reviewing.

Review process

ICER research papers are reviewed using an anonymous process managed through EasyChair. The process involves the following activities and conflict of interest management:

  • Abstracts. Authors initially submit an abstract by the abstract submission deadline, with the final deadline for full papers one week later.
  • Conflicts of interest. Authors are required to declare conflicts of interest with all reviewer, meta-rereviewers, and program chairs.
  • Bidding. Following the abstract submission deadline, review committee and program committee members bid on papers for which they have sufficient expertise to evaluate. These bids help the authors assign reviewers.
  • Conflicts of interest. All reviewers, meta-reviewers, and program chairs who have a conflict of interest with a paper must register that conflict in EasyChair, after which they are excluded excluded from all future evaluation, discussion, and decisions of that paper.
  • Submissions. Authors submit final versions of their submissions.
  • Desk rejects. The program chairs inspect all submissions for desk rejects, including papers that violate anonymization requirements, length restrictions, or plagiarism policies. Authors of desk rejected papers are notified immediately.
  • Conflicts of interest. Program chairs with conflicts are excluded from deciding on those papers, leaving the decision to the other program chair.
  • Assignments. Each paper is assigned to at least three reviewers and exactly one program committee member. Program chairs do all assignments together.
  • Conflicts of interest. Program chairs with conflicts are excluded from assigning reviewers to any papers for which they have a conflict. Assignments in EasyChair can only be made by the program chair without a conflict.
  • Reviews. Assigned reviewers submit their anonymous reviews through EasyChair by the review deadline, evaluating each of their papers against the review criteria listed above. Reviewers are not asked to make an accept/reject decision, only to evaluate the papers against the criteria. This ensures that there is still room for changing opinions through discussions.
  • Conflicts of interest. Reviewers, meta-reviewers, and program chairs with conflicts cannot see any of the reviews of their papers during this process.
  • Discussion. After the reviewing period, the program committee member assigned as meta-reviewer asks the reviewers to read the other reviewers’ reviews and begin a discussion about any disagreements that arise.
  • Conflicts of interest. Reviewers, meta-reviewers, and program chairs with conflicts cannot see any of the discussions of their papers during this process.
  • Recommendation. After the discussion, the meta-reviewer uses the reviews, the discussion, and their own evaluation of the work to write a recommendation of whether to accept or reject the work, or whether to discuss it at the PC meeting. Meta-reviewers also decide whether to nominate the paper for the Chair’s award.
  • Conflicts of interest. Reviewers, meta-reviewers, and program chairs with conflicts cannot see any of the recommendations or meta-reviews of their papers during this process.
  • PC meetings. “Borderline” submissions (those for which there is no clear outcome, accept or reject) will be discussed in a series of videoconference meetings of PC members. Meetings will be convened as required (a maximum of three meetings of three hours each) during the week of May 8th to May 15th, at times to be decided (so as to best to accommodate committee members).  PC members will be expected to attend one or more of these meetings. The role of the meetings is to reach consensus (by vote if required) on the outcome for each borderline submission. After a decision is made in each case, the responsible PC member (meta-reviewer) will add a summary of the discussion at the end of their meta-review, explaining the rationale for the final decision.
  • Conflicts of interest. Because the discussion will happen in EasyChair, no meta-reviewers or program chairs conflicted on a paper will be able to see or participate in the discussion.
  • Decisions. After the PC meeting, the program chairs notify all authors of the decisions about their papers no later than the notification date listed in the CFP.
  • Conflicts of interest. Program chairs cannot change the outcome of a decision after the PC meeting.
  • Papers for which any program chair has a conflict of interest are not eligible for a Chair’s Award.

Publication

Once papers are accepted and finalized for publication, the official publication date is the date the proceedings are made available in the ACM Digital Library. This date will be up to two weeks prior to the first day of the conference. The official publication date affects the deadline for any patent filings related to published work.

The ACM has recently revised its authorship policy in a way that might impact some ICER authors. In summary, any person listed as an author on a paper must (1) have made substantial contributions to the work, (2) have participated in drafting/revising the paper, (3) be aware that the paper has been submitted, and (4) agree to be held accountable for the content of the paper. The revision is intended to allow enforcement of plagiarism sanctions, but it could impact people who work in large, collaborative research groups, and on postgraduate advisors who have not contributed directly to a paper. Prospective authors should read the new policy. You should also ensure that your submissions abide by the ACM Conflict of Interest Policy. Aligned with this is also the ACM Publications Policy on the Withdrawal, Correction, Retraction, and Removal of Works from ACM Publications and ACM DL.