Blogs (4) >>
SIGCSE TS 2024
Wed 20 - Sat 23 March 2024 Portland, Oregon, United States
SIGCSE TS 2024 (series) /

Reviewer Information and Guidelines

This page provides some general information for reviewers. Reviewers can find additional information in the “Instructions for Reviewers” tab on the individual tracks.

To volunteer to review for SIGCSE TS 2024, please visit https://bit.ly/review-SIGCSE2024. If you are not sure whether or not you have already volunteered to review, please send email to program@sigcse2024.sigcse.org.

Table of Contents

Ethical Reviewing

We expect SIGCSE TS reviewers to adhere to high standards. Among other things, this means that Reviewers are expected to take the time to read submissions carefull and to write serious, professional reviews whose primary goal is to improve the submission, whether or not the submission is recommended for acceptance. Reviewers should make sure that they are appropriately familiar with the domain and techniques used in the submission. Reviewers should also address issues in their reviews raised by Program Chairs, Track Chairs, Associate Program Chairs, and fellow Reviewers.

Roles in the Review Process

  • Reviewers write reviews of their assigned submissions, evaluating them against the review criteria.
  • Associate Program Chairs (APCs) write meta-review for their assigned submissions and provide a recommendation (accept/reject) and feedback to the Program Chairs. Currently, SIGCSE TS uses APCs only for papers and posters.
  • Track Chairs take responsibility for the non-paper tracks at SIGCSE TS.
  • Program Chairs make the final decisions on the program based on recommendations from the APCs (for papers) and from track chairs (for other tracks).

SIGCSE TS has three Program Chairs, each of whom serves a two-year term. Nominations for Program Chairs are solicited by the SIGCSE TS steering committee, which makes recommendations to the SIGCSE Board. Program Chairs are appointed by the SIGCSE board.

The Program Chairs solicit applications for Track Chairs. Program Chairs make recommendations to the SIGCSE TS Steering Committee for Track Chairs, who are normally appointed for a two-year term.

The Program Chairs invite the Reviewers and APCs. Program Chairs and Track Chairs work together to appoint Reviewers and APCs. The number of submissions per Reviewer/APC generally depends on the number of volunteers and the size of the submissions pool.

The goals for each paper submission to receive at least three reviews and a meta-review. All reviews are submitted through the submission system. In EasyChair, Reviewers are considered “Ordinary PC members” and APCs are considered “Senior PC members”.

Submission and Review System

The review process for SIGCSE TS 2024 will be done using the EasyChair submission system (https://easychair.org/my/conference?conf=sigcsets2024) . Reviewers will be invited to join/login into EasyChair, update their profile, and select 3-5 topics that they are most qualified to review. To do so, reviewers select SIGCSE TS 2024 > Conference > My topics from the menu and select at most 5 topics. More topics make it harder for the EasyChair system to make a good set of matches. Reviewers also identify their Conflicts of Interest by selecting SIGCSE TS 2024 > Conference > My Conflicts.

Conflicts of Interest

SIGCSE TS takes conflicts of interest, both real and perceived, quite seriously. The conference adheres to the ACM conflict of interest policy (https://www.acm.org/publications/policies/conflict-of-interest) as well as the SIGCSE conflict of interest policy (https://sigcse.org/policies/COI.html). These state that a paper submitted to the SIGCSE TS is a conflict of interest for an individual if at least one of the following is true:

  • The individual is a co-author of the paper
  • A student of the individual is a co-author of the paper
  • The individual identifies the paper as a conflict of interest, i.e., that the individual does not believe that they can provide an impartial evaluation of the paper.

The following policies apply to conference organizers:

  • The Program Chairs are not allowed to submit to any track.
  • The chairs of any track are not allowed to submit to that specific track.
  • All other conference organizers are allowed to submit to any track.
  • All reviewers (PC members) and meta-reviewers (APC members) are allowed to submit to any track.

No reviewer, meta-reviewer, or chair with a conflict of interest in the paper will be included in any evaluation, discussion, or decision about the paper. It is the responsibility of the reviewers, meta-reviewers, and chairs to declare their conflicts of interest throughout the process. The corresponding actions are outlined below for each relevant step of the reviewing process. It is the responsibility of the chairs to ensure that no reviewer or meta-reviewer is assigned a role in the review process for any paper for which they have a conflict of interest.

Anonymity

In all cases, reviewers are anonymous to authors and to each other. Wherever possible, authors are also anonymous to reviewers. However, for certain tracks (Panels, Special Sessions, Workshops, Birds-of-a-Feather, Demos, and ACM Student Research Competition), authors are not anonymous. In most cases, this is because the identity of the presenters will likely have a significant effect on the value of or interest in a session.

We use different review processes for round-one single-anonymous submissions (Panels, Special Sessions, and Workshops) and round-two single-anonymous submissions.

Dual-Anonymous Review Process

In the dual-anonymous review process, Reviewers are unaware of the author identities, and reviewers are anonymous to each other and to the authors.

Authors must submit ONLY an anonymized version of the submission. The goal of the anonymized version is to, as much as possible, provide the author(s) of the submission with an unbiased review. The anonymized version should have ALL mentions of the authors removed (including author’s names and affiliation plus identifying information within the body of the submission such as websites or related publications). However, authors are reminded to leave sufficient space in the submitted manuscripts to accommodate author information either at the beginning or end of the submission.

LaTeX/Overleaf users are welcome to use the anonymous option but are reminded that sufficient room must exist in the submission to include all author blocks when that option is removed. Authors may choose to use placeholder text in the author information block, but we encourage authors to use obviously anonymized placeholders like “Author 1”, “Affiliation 1”, etc.

Self-citations need not be removed if they are worded so that the reviewer doesn’t know if the writer is citing themselves. That is, instead of writing “We reported on our first experiment in 2017 in a previous paper [1]”, the writer might write “In 2017, an initial experiment was done in this area as reported in [1].

The SIGCSE TS 2024 dual-anonymous review process does not have a rebuttal period for authors to respond to comments, and all acceptance decisions are final.

Single-Anonymous Review Process

In the single-anonymous review processauthor identities are known to reviewers, but reviewers are anonymous to each other and to the authors. Single-anonymous submissions should include author names and affiliations.

In most cases, after submitting reviews, reviewers in single-anonymous tracks are encouraged to discuss their reviews with their Track Chairs or APCs.

Neither SIGCSE TS 2024 single-anonymous review process has a rebuttal period for authors to respond to comments. All acceptance decisions are final.

New Round-One Single-Anonymous Review Process for SIGCSE TS 2024

In response to issues identified in certain single-anonymous reviews, SIGCSE TS 2024 is introducing a new review process for Panels, Special Sessions, and Workshops. Reviewers will be asked to review approximately six proposals. Initial reviews will consist of a single rating (Recommend acceptance, Unsure, Do not recommend acceptance) and a short rationale for that rating. After the initial reviews, the track chairs will propose a slate to accept. In putting together the slate, the chairs will consider ratings, comments (and appropriateness thereof), and the balance of topics. Reviewers will then discuss both the slate of reviews and the complete set of submissions, proposing substitutions as appropriate. Track chairs will then make the determination of the final slate and will write meta-reviews for the authors, based on the initial comments and the discussion.

We expect that the broader discussion of the slate will provide a more equitable and transparent review process and that the replacement of individual reviews with meta-reviews will better address issues of potential bias in individual reviews.

General Review Guidelines

Reviewers provide high-quality reviews for submissions to provide authors with feedback so they may improve their work for presentation or future submissions. As such, please ensure that all criticism is phrased in a constructive manner.

Your overall recommendation should focus on the significance and relevance, anticipated interest, and quality of the proposal.

We strongly recommend that you prepare your review in a separate document; EasyChair has been known to time out.

While your review text should clearly support your scores and recommendation, please do not include your preference for acceptance or rejection of a submission in the feedback to the authors. Instead, use the provided radio buttons to make a recommendation (the authors will not see this) based on your summary review and provide any details that refer to your recommendation directly in the confidential comments to the APC or track chairs. Remember that as a reviewer, you will only see a small portion of the submissions, so one that you recommend for acceptance may be rejected when considering the other reviewer recommendations and the full set of submissions.

Discussion

The discussion period provides an opportunity for the APCs or Track Chairs to address discrepancies in reviews, garner more information, and determine a final recommendation to the Program Chairs.

At the start of the discussion period, reviewers should read the other reviews of each submission and consider whether they have an effect on your own review. If so, please update your own review. If you observe issues in other reviews, such as inappropriate comments, please raise them with the other reviewer and/or the APC or Track Chairs. If you have a difference of opinion on a submission, you might raise questions in the discussion forum for the submission on EasyChair.

During the discussion period, you should respond to the comments from other reviewers and the APC or Track Chairs.

Remember that our goal is to provide authors with clear and useful feedback on improving their submission and its presentation and to provide the Program Chairs with appropriate information to allow them to select an appropriate slate of submissions to put together the program. Even submissions that are likely to be accepted benefit from feedback to the authors to improve the work further and to program chairs as to why they should be accepted.

Recalcitrant Reviewers

Reviewers who don’t submit reviews, have reviews with limited constructive feedback, do not engage effectively in the discussion phase, or submit inappropriate reviews will be removed from the reviewer list (as per SIGCSE policy). Recalcitrant reviewers will be informed of their removal from the reviewer list. Reviewers with repeated offenses (two within a three-year period) will be removed from SIGCSE reviewing for three years.