SIGCSE 2018 Review Process for Papers

SIGCSE papers are reviewed using a double-blind process managed through EasyChair. Authors initially submit an abstract by the abstract submission deadline (Aug 25th), with the final deadline for full papers one week later (Sept. 1, 2017). Following the abstract submission deadline, the reviewers (EasyChair refers to reviewers as “ordinary PC members”) and associate program committee (APC) members (EasyChair refers to APC as “senior PC members”) bid on papers of interest and register conflicts. This process is used in an effort to best match papers with reviewer’s and APC’s expertise. Each paper is assigned to at least three reviewers and exactly one APC.

Assigned reviewers submit their anonymous reviews through EasyChair by the review deadline (Sept. 20, 2017), making a recommendation along with a rationale using the conference review form in EasyChair. During this review period, the APC assigned will encourage reviewers to complete their reviews by the deadline. During the meta-review period (Sept. 20-26), the APCs and reviewers assigned to a paper engage in anonymous discussion through the EasyChair system to ascertain whether or not the paper should be accepted for publication. By the decision deadline (Sept. 26), the APCs prepare a meta-review, making an accept/reject recommendation along with a rationale, using the conference meta-review form.

Meta-Review Discussion:

  • The goal is not to have reviewers change or update their scores, though that might happen as a by product of the discussion.
  • The goal is to reach an agreement on the quality of the submission. For example, one reviewer might find objection with some premises of the paper and give the paper a low score. Another reviewer might excuse that limitation and find of high quality the results and give the paper a high score. Both reviews are valid, presumably, and thus their scores should not be updated. But their reviews (and possibly the meta-review) should highlight the tradeoffs that result from this discussion, and come up with an agreeable decision to both reviewers.
  • In a few rare cases, the reviewers will have opposite views and the meta-reviewer should capture the essence of all reviews and leave the recommendation as neutral.
  • At no point reviewers should feel forced to change their reviews, scores, or points of view in this process.

Dates for notification of acceptance and camera-ready submission can be found on the detail page for each specific submission type.

CS Education Research Paper Track

Papers submitted to the CS Education Research track describe an empirical computing education project.

CS Education Research papers should adhere to rigorous standards, describing hypotheses, methods, and results as is typical for research studies. These normally focus on topics relevant to computing education with emphasis on educational goals and knowledge units/topics relevant to computing education with statistical rigor; methods or techniques in computing education; evaluation of pedagogical approaches; and studies of the many different populations that are engaged in computing education, including (but not limited to) students, instructors, and issues of gender, diversity, and underrepresentation.

For a typical paper in this track, here are some key factors to include (as an author) and to look for (as a reviewer):

  1. Are there one or more clearly stated research questions? Since the rest of the paper will be organized around these, it’s often good to put them in the abstract and in the first section of the paper.
  2. Are the questions of interest to the SIGCSE audience?
  3. Related work in computing education
    • Is the relevant work in computing education included? If not, a good review must give references to missing material. Simply saying “The related work section is incomplete” is not enough.
    • Do the authors clearly describe the relationship between the previous work and the current research questions? In what ways does the current project build on the previous work, and how is it different?
  4. Related work in educational theory
    • Is the project based in educational theory?
    • Is the theory described clearly, with appropriate citations?
    • Is the theory’s relationship to the current project clearly described?
  5. Is the data gathering sufficiently clearly described that the reader could reproduce it? Some key information to include:
    • About the data: why this particular type of data is relevant to your research questions
    • About the participants: how many, what was their background (are they instructors, students, alumni, etc.); what if any formal coursework have they had in computing; how many were men and how many women; and any other factors that are relevant to the author’s project
    • About the person(s) gathering the data: What is their relationship to the participants? For example, if the data were collected from students in a class, was the instructor one of the researchers or not?
    • About the data gathering process: did the project use surveys, interviews, samples of student work, other; If surveys or interviews, exactly what questions were asked.
  6. Is the data analysis sufficiently clearly described that the reader could reproduce it?
    • What methodology was used?
    • Is the methodology described, with an appropriate citation?
    • Is the implementation of the methodology clearly enough described? How many people were involved? What process was used to resolve any disagreements?
  7. Is the analysis methodology something new to computing education research that might be a contribution in itself?
  8. Are the results of the analysis clearly summarized?
  9. Are the results thoroughly discussed, including:
    • Their relationship to the research questions
    • Their relationship to previous work
    • Any threats to validity
    • The implications of the results for future research
    • The implications of the results for teaching

Experience Reports and Tools Paper Track

Experience Reports and Tools papers should carefully describe a computer science education intervention and its context, and provide a rich reflection on what worked, what didn’t, and why. This track accepts experience reports, teaching techniques, and pedagogical tools. All papers in this track should provide enough detail so that others could adopt the new innovation

For a typical paper in this track, here are some key factors to include (as an author) and to look for (as a reviewer):

  1. Are there one or more clearly stated goals in this paper? Since the rest of the paper will be organized around these, it’s often good to put them in the abstract and in the first section of the paper.
  2. Is the experience or tool of interest to the SIGCSE audience?
  3. Related work in computing education
    • Is the relevant work in computing education included? If not, a good review must give references to missing material. Simply saying “The related work section is incomplete” is not enough.
    • Do the authors clearly describe the relationship between the previous work and the current research questions? In what ways does the current project build on the previous work, and how is it different?
  4. Are the observations and/or findings from the experience or the use of a tool clearly summarized?
  5. Are the findings thoroughly discussed, including:
    • Their relationship to previous work
    • The implications of the results for future use
    • The implications of the results for teaching
    • Information on how to adopt or adapt teaching techniques and/or pedagogical tools in other contexts or institutions.

New Curricula, Programs, Degrees and Position Paper Track

New curricula, programs, degrees and position papers. Papers about curricula, programs and degrees should describe the motivating context before the new initiative was undertaken, what it took to put the initiative into place, what the impact has been, and suggestions for others wishing to adopt it. Position papers are meant to engender fruitful academic discussion by presenting a defensible opinion about a CS education topic, substantiated with evidence.

  1. Is the innovation clearly stated? Since the rest of the paper will be organized around this, it’s often good to put it in the abstract and in the first section of the paper.
    • Description of the problem or need being addressed.
  2. Is the curricular innovation or position paper of interest to the SIGCSE audience?
  3. Related work in computing education
    • What prior solutions to this problem exist?
    • Is the relevant work in computing education included? If not, a good review must give references to missing material. Simply saying “The related work section is incomplete” is not enough?
    • Do the authors clearly describe the relationship between the previous work and the current research questions? In what ways does the current project build on the previous work, and how is it different?
  4. Examination for discussion
    • How is the curricular innovation or position paper addressing this problem or need?
    • How is the curricular innovation or position paper different from previous ideas?
  5. Future Success Indicators
    • How could the curricular innovation or proposed idea be assessed if adopted or implemented?
    • In what context can the curricular innovation or proposed idea be used (large research institutions, community colleges, high schools)?
    • How difficult would the curricular innovation or proposed idea be to adopt it? For example, the human and financial resources needed.