This form can also be downloaded as a DOCX or a PDF file.
Related Work
Discussion of related work (*). Discussing related work is important for several reasons. It sets the stage for the research and allows the authors to clarify their own contribution. What have other researchers done? What kind of gap is there in the previous research? How does the work presented in the paper fill that gap or improve or extend previous work and results?
| 5 | all relevant work discussed and cited, and relationship to submission clearly and thoroughly described |
| 4 | covers key related work; its relationship to submission is described, but could be extended further |
| 3 | some references missing, or their relationship to submission not clearly described |
| 2 | several important or key references missing, and relationship of references to submission not apparent |
| 1 | no discussion of related work |
Rationale for your rating (free text)
Theoretical Basis
Theoretical basis for the paper (*). Educational sciences, psychology, and other social sciences provide many theories, models, and conceptual frameworks, which also support computing education research. They can provide concepts and vocabulary for discussing research. They can guide research design and interpretation of empirical results and provide a lens for interpreting the results. They can provide stronger arguments in discussing the research, and they can sometimes be used to build and test hypotheses for empirical research. However, within computing education research, there is not always an obvious theoretical basis; for example, because the work may concern the development and use of a new tool or technology, or it may focus on collecting and analyzing data that is highly computing-specific, and so far no obvious theoretical basis has been developed to address it. If you have suggestions for other or better theoretical framework(s), please provide this feedback for the authors.
| 5 | clear and strong theoretical basis, well documented with citations and clearly applied in the research |
| 4 | theoretical basis presented, with some citations and argument for how it is applied in the research |
| 3 | there is a theory presented, but its relevance to the research is weak |
| 2 | there is a theory presented, but it is vague and has no clear relevance to the research |
| 1 | no obvious theory presented, even though some theories could be applied here (which?) |
| 0 | not applicable: theoretical basis is not relevant for the work presented in this paper; please give your arguments below |
Rationale for your rating (free text)
Methodology, exposition, results, and conclusion
Research methodology (*). Research methodology provides a holistic approach to empirical research design for addressing the research questions, including arguments for selecting data collection, methods of analysis, interpretation of results, and discussion of limitations of the work. Note 1. The conference also accepts papers which focus on a theoretical discussion of some topic related to computing education. Such papers should clearly argue why empirical work is missing and re-interpret previously published empirical results, or should collect and analyse data to build a proof of concept of the new theoretical contribution. Note 2. ICER accepts studies which replicate previous empirical work. In these cases it is important to clearly justify the methodology so that comparison with previous work is possible. Note 3. ICER values work using quantitative, qualitative, or mixed methods, and recognizes that both formative studies and summative evaluations can make crucial contributions to the field.
| 5 | research approach and/or methods well-suited for the research questions/hypotheses and supported with exemplary arguments |
| 4 | research approach and/or methods well-suited and appropriately applied |
| 3 | research approach and/or methods appropriate but there are flaws in applying them |
| 2 | research approach and/or methods inappropriate for research objectives |
| 1 | research approach and/or methods cannot be identified |
| 0 | not applicable: the paper does not present empirical work, and explains why |
Data collection and analysis methods (*). Score the paper’s collection and analysis of data.
| 5 | exemplary presentation of data collection and analysis methods with excellent argumentation of the choices. |
| 4 | data collected and analyzed; methods clear and thoroughly described |
| 3 | data collected and analyzed, but some aspects of methods unclearly described |
| 2 | data collected and analyzed, but methods are not described |
| 1 | unclear what data was collected or analyzed, if any |
| 0 | not applicable: the paper does not present empirical work, and explains why |
Discussion of results and conclusions (*). How well have the authors interpreted their findings?
| 5 | incisive interpretation of results and limitations |
| 4 | good interpretation of results; limitations considered |
| 3 | plausible interpretation of results |
| 2 | questionable interpretation of results |
| 1 | unjustifiable interpretation of results |
| 0 | not applicable: the paper does not present any discussion or empirical results |
Rationale for ratings of methodology, research methods, results, and conclusions. (*).
Explain why you gave the ratings you did for the three fields above. (free text)
Contribution, Relevance, and Significance
Contribution and relevance to the international computing education research field (*). The paper should make it clear how it advances the international computing education research field. This typically requires comparison of results to previous related work and/or theoretical basis. Moreover, the authors should express how their work contributes to the audience beyond their local context. Finally, ICER is an international conference and therefore all problems, concepts, or terms which are culturally situated should be made clear in the paper; otherwise international readers might not comprehend the paper. Some examples of such local concepts are AP exam in the US, and ECTS credits in Europe
.
| 5 | a major and significant contribution to computing education research that explicitly presents results in a manner directly applicable to international research contexts |
| 4 | a clear contribution to computing education research |
| 3 | minor contribution to computing education research or contribution is bound to a local context, perhaps with the promise of more to come |
| 2 | no obvious contribution to computing education research, but the promise of future value |
| 1 | contributes little or nothing to computing education research |
Rationale for your rating (free text)
Writing and Expression
Writing and expression. Evaluate the writing and expression in the paper. The text field is not mandatory, but please use it if you wish to make specific comments on the writing.
| 5 | well written and expressed |
| 3 | mostly well written; recommendations below for changes if accepted |
| 1 | not well written, with frequent grammatical, stylistic, or structural problems |
Rationale for your rating (free text)
Additional Comments for Authors
Suggestions or other comments. Optional: if you would like to make any further comments for the authors, please do so here. (free text)
Overall Recommendation
Overall evaluation (*). The overall recommendation is not the average value of your observations above, but it should generally reflect them. For example, even if you like the paper very much, do not recommend Strong accept if you have pointed out several weaknesses in it. Furthermore, even weak papers have some positive aspects. Please point these out to the authors in this field.
| 6 | Strong accept |
| 5 | Accept |
| 4 | Weak accept |
| 3 | Weak reject |
| 2 | Reject |
| 1 | Strong reject |
Rationale for your rating (free text)
Confidential Comments
Confidential remarks for reviewers, meta-reviewers, and program co-chairs. Optional: comments written here will not be seen by the submission’s authors. (free text)