PapersSIGCSE TS 2024
Papers describe an educational research project, classroom experience, teaching technique, curricular initiative, or pedagogical tool in the computing content domain. All papers submitted to the SIGCSE TS should be original work that complies with the ACM authorship policies. SIGCSE TS considers papers in three distinct tracks, each with their own unique expectations. See further details below.
Paper Tracks
Please ensure that you submit your paper to the correct paper track by reading the the Reviewing Guidelines. Papers will be reviewed for the track they are submitted to and will not be moved between tracks. Any submissions made to more than one track will be desk rejected from both tracks.
- Computing Education Research. The primary purpose of Computing Education Research (CER) papers is to advance what is known about the teaching and learning of computing. CER papers are reviewed relative to the clarity of the research questions posed, the relevance of the work in light of prior literature and theory, the soundness of the methods to address the questions posed, and the overall contribution. Both qualitative and quantitative research is welcomed, as are replication studies and papers that present null or negative results.
- Experience Reports and Tools. The primary purpose of Experience Reports and Tools (ERT) papers is observational in nature, and ERT papers should carefully describe the development and use of a computing education approach or tool, the context of its use including the formative data collected, and provide a rich reflection on what did or didn’t work, and why. ERT contributions should be motivated by prior literature and should highlight the novelty of the experience or tool presented. ERT papers differ from CER papers in that they frame their contributions to enable adoption by other practitioners, rather than focusing on the generalizability or transferability of findings, or threats to validity.
- Position and Curricula Initiative. The primary purpose of Position and Curricula Initiative (PCI) papers is to present a coherent argument about a computing education topic, including, but not limited to curriculum or program design, practical and social issues facing computing educators, and critiques of existing practices. PCI papers should substantiate their claims using evidence in the form of thorough literature reviews, analysis of secondary data collected by others, or another appropriate rhetorical approach. In contrast to CER papers, PCI papers need not present original data or adhere to typical qualitative or quantitative research methods. PCI papers differ from ERT papers in that they do not necessarily report on individual experiences, programs or tools, but rather they may focus on broader concerns to the community.
Papers submitted to all tracks should address one or more computing content topic. Authors will be asked to select between 3 and 7 topics from this list at the time of submission. Papers deemed outside the scope of symposium by the program chairs will be desk rejected without review.
Authors submitting work to SIGCSE TS 2024 are responsible for complying with all applicable conference authorship policies and those articulated by ACM. If you have questions about any of these policies, please contact program@sigcse2024.sigcse.org for clarification prior to submission.
New for 2024: ACM has made a commitment to collect ORCiD IDs from all published authors (https://authors.acm.org/author-resources/orcid-faqs). All authors on each submission must have an ORCiD ID (https://orcid.org/register) in order to complete the submission process. Please make sure to get your ORCiD ID in advance of submitting your work.
Presentation Modality
Papers at SIGCSE TS 2024 can be presented either in-person using a traditional paper session or online via a synchronous Zoom session with Q/A. Pre-recorded videos will NOT be required. The online zoom presenters will be scheduled to present synchronously during the conference days just like the in-person presenters. Authors of accepted submissions must commit to one of these two presentation modalities in a timely manner to facilitate conference planning. Registration rates for online presenters are likely to be comparable to those for in-person attendees and higher than that of online-only attendees, which will help offset the additional costs of supporting online presentation. Further instructions and information will be provided in acceptance notifications.
Deadlines and Submission
Papers submitted to SIGCSE TS 2024 follow a two-step submission process. The first step requires that authors submit all paper metadata and a plain text abstract in EasyChair no later than Friday, 11 August 2023. This data is used to allow reviewers to bid on potential papers to maximize the match of reviewer expertise to paper content. To help the bidding and reviewing process, please submit an abstract that is as close to the finished version as possible. The Program Chairs reserve the right to desk reject abstracts that do not contain content that can help a reviewer during bidding.
The second step of the paper submission process is to upload the final anonymized PDF of the full paper for review. This must be completed no later than Friday, 18 August 2023. Authors who fail to submit an abstract by the first deadline will not be permitted to submit a full PDF.
Important Dates
| Abstact Due Date | Friday, 11 August 2023 | |
| Abstract Due Time | 23:59 AoE (Anywhere on Earth, UTC-12h) | |
| Due Date | Friday, 18 August 2023 | |
| Due Time | 23:59 AoE (Anywhere on Earth, UTC-12h) | |
| Submission Limits | 6 pages + 1 page only for references | |
| Notification to Authors | Monday, 2 October 2023 tentative | |
| Submission Link | https://easychair.org/conferences/?conf=sigcsets2024 | |
| Session Duration | 20 minutes | |
Instructions for Authors
Authors may find it useful to read the Instructions for Reviewers and the Review Forms to understand how their submissions will be reviewed. Also note that when submitting, you will need to provide between 3-7 related topics from the Topics list under Info.
Abstracts
All papers must have a plain-text abstract of up to 250 words. Abstracts should not contain subheadings or citations. The abstract should be submitted in EasyChair along with paper metadata, and the same text should be included in the PDF version of the full paper at the appropriate location.
Submission Templates
SIGCSE TS 2024 is NOT participating in the new ACM TAPS workflow, template, and production system.
All paper submissions must be in English and formatted using the 2-column ACM SIG Conference Proceedings format and US letter size pages (8.5x11 inch or 215.9 x 279.4mm).
Here is an annotated PDF example that has some notes/tips and shows the required sections.
Page Limits: Papers are limited to a maximum of 6 pages of body content (including all titles, author information, abstract, main text, tables and illustrations, acknowledgements, and supplemental material). One additional page may be included which contains only references. If included, appendix materials MUST NOT be present on the optional references page.
MS Word Authors: Please use the interim Word template provided by ACM.
LaTeX Authors:
- Overleaf provides a suitable two-column sig conference proceedings template.
- Please do not use the
anonymousdocument class option, as counter-intuitive as that sounds. We’d like to ensure that author blocks appear in the submission, and that option removes them. - Other LaTeX users may alternatively use the ACM Primary template, adding the “
sigconf” format option in thedocumentclassto obtain the 2-column format. - NOTE: The default LaTeX template text shows appendix materials following the references. SIGCSE TS 2024 does not permit appendices on the optional page allotted for references. Authors must include all relevant content within the 6 body pages of the paper. References are the ONLY thing that can be added on page 7.
Requirements for Double Anonymous Review Process: At the time of submission all entries must include blank space for all anonymous author information (or anonymized author name, institution, location, and email address), followed by an abstract, keywords, CCS Concepts, placeholders for the ACM Reference Format and copyright blocks, and references. For anonymized submissions, all blank space necessary for all author information must be reserved under the Title, or fully anonymized text can take its place (e.g. 4 lines containing Author1, Author1Institution, Author1Location, anon1@university.edu). In addition, please leave enough blank space for what you intend to include for Acknowledgements but do not include the text, especially names and granting agencies and grant numbers. Acknowledgements should be included in the first 6 pages.
Other requirements: Please provide a separate block for each author, including name, email, institution, location, and country, even if authors share an institution.
Desk Rejects: Papers that do not adhere to page limits or formatting requirements will be desk rejected without review.
Accessibility: SIGCSE TS 2024 authors are strongly encouraged to prepare submissions using these templates in such a manner that the content is widely accessible to potential reviewers, track chairs, and readers. Please see these resources for preparing an accessible submission.
Double Anonymized Review
Authors must submit ONLY an anonymized version of the paper. The goal of the anonymized version is to, as much as possible, provide the author(s) of the paper with an unbiased review. The anonymized version must have ALL mentions of the authors removed (including author’s names and affiliation plus identifying information within the body of the paper such as websites or related publications). However, authors are reminded to leave sufficient space in the submitted manuscripts to accommodate author information either at the beginning or end of the paper. LaTeX/Overleaf users are welcome to use the anonymous option, but are reminded that sufficient room must exist in the 6 body pages to include all author blocks when that option is removed. Authors may choose to use placeholder text in the author information block, but we encourage authors to use obviously anonymized placeholders like “Author 1”, “Affiliation 1”, etc.
Self-citations need not be removed if they are worded so that the reviewer doesn’t know if the writer is citing themselves. That is, instead of writing “We reported on our first experiment in 2017 in a previous paper [1]”, the writer might write “In 2017, an initial experiment was done in this area as reported in [1].
As per ACM guidelines, authors may distribute a preprint of their work on ArXiv.org. However, to ensure the anonymity of the process, we ask that you not publish your work until after you receive the accept/reject notice. If particular aspects of your paper require earlier distribution of the preprint, please consider changing the title and abstract so that reviewers do not inadvertently discover your identity.
Submissions to the Papers tracks are reviewed with the dual-anonymous review process. The reviewers and meta-reviewers (i.e. associate program chairs or APCs) are unaware of the author identities, and reviewers and APCs are anonymous to each other and to the authors.
The reviewing process includes a discussion phase after initial reviews have been posted. During this time, the reviewers and APC can examine all reviews and privately discuss the strengths and weaknesses of the work in an anonymous manner through EasyChair. Following discussion, the APC shall draft a meta-review that holistically captures the group position on the paper, incorporating views raised in the reviews and during the discussion phase.
The SIGCSE TS 2024 review process does not have a rebuttal period for authors to respond to comments, and all acceptance decisions are final.
ACM Policies
By submitting your article to an ACM Publication, you are hereby acknowledging that you and your co-authors are subject to all ACM Publications Policies, including ACM’s new Publications Policy on Research Involving Human Participants and Subjects (https://www.acm.org/publications/policies/research-involving-human-participants-and-subjects). Alleged violations of this policy or any ACM Publications Policy will be investigated by ACM and may result in a full retraction of your paper, in addition to other potential penalties, as per ACM Publications Policy.
ORCiD ID
ACM has made a commitment to collect ORCiD IDs from all published authors (https://authors.acm.org/author-resources/orcid-faqs). All authors on each submission must have an ORCiD ID (https://orcid.org/register) in order to complete the submission process. Please make sure to get your ORCiD ID in advance of submitting your work. (If EasyChair does not request the ORCiD ID for your coauthors, you do not need to find a way to enter one.)
Author Checklist
Additional details are in the instructions for authors.
Getting ready
- Make sure that all authors have obtained an ORCiD identifier. These identifiers may be required for paper submission.
- Identify at least one author who is willing to review for the symposium. Have that author or those authors sign up to review at https://bit.ly/review-SIGCSE2024. (If they’ve done so already, there is no need to fill out the form a second time.)
- Download the appropriate template. Check this annotated PDF example that has some notes/tips and shows the required sections.
- Select the appropriate paper track for your submission.
- Review the additional resources for the track.
- Review the instructions for reviewers and the review forms to see what reviewers will be looking for in your submission.
- Look at the list of topics in the Info menu on this site or on EasyChair and pick 3-7 appropriate topics for your submission. This helps in matching reviewers’ expertise with submissions and is different from the next item.
- Make certain that you have entered CCS concepts in your paper by choosing them from the ACM Computing Classification System site.
- Look at the EasyChair submission page to make sure you’ll be prepared to fill everything out. Note that you are permitted to update your submission until the deadline, so it is fine to put draft information there as you get ready.
The abstract on EasyChair
Note: EasyChair does not let you save incomplete submission forms. Please fill out all of the fields in one sitting and save them. After that, you can continue to update the information in the fields and your submission until the deadline.
- Select the appropriate paper track for your paper
- Submit a 250-word abstract by 11:59 p.m. AOE, Friday, 11 August 2023.
- IMPORTANT: as you enter the author names in EasyChair consider the order. Author lists can NOT be modified (this includes add/remove/reorder)
The paper on EasyChair
- Use an appropriate template.
- Ensure that your submission is appropriately anonymized.
- Ensure that your submission is accessible. See accessibility tips for authors for further details.
- Ensure that your submission is no more than six pages (plus an additional page for references ONLY).
- If you add Acknowledgements, anonymize the text and include them in the six pages. (they can NOT be on the additional page)
- Submit the final version of your paper by 11:59 p.m. AOE, Friday, 18 August 2023.
Post-Acceptance and Presentation Information
Presentation Modality: Due Friday, 13 October 2023
Authors for all accepted papers must select a mode for presenting at the symposium (online or in-person). The first corresponding author on each paper should receive a Qualtrics survey by email shortly after acceptance notifications are sent. This survey should be completed only once per accepted paper.
Presentation modality selection is required by 13 October 2023. If authors do not submit a modality choice by the deadline, the paper will default to online presentation modality and will not be assigned to an in-person session.
Registration: Tentatively Starts Wednesday, 1 November 2023
In order for your paper to be presented at the symposium and included in the proceedings, at least one author must register for the conference. Please let us know immediately if you or your co-authors are unable to present your paper at the symposium so we can withdraw it.
Camera-Ready: Due Wednesday, 14 December 2023
Authors should carefully consider the reviews when preparing final CAMERA-READY submissions. A camera-ready PDF must be submitted to Sheridan Communications for inclusion in the conference proceedings.
- Authors can find initial instructions for preparing final camera-ready documents here: http://www.scomminc.com/pp/acmsig/sigcse2023.htm
- We also remind authors to review the accessibility tips to ensure the symposium content is widely usable for all parties.
Optional Video Presentations: Tentatively Due Jan 2023
Authors opting to provide the OPTIONAL video for the ACM DL as described in the camera-ready instructions, must check “YES” for being recorded on the ACM rights review form. If that option is not checked, the video will not be included in the ACM DL. Those who check “YES” will be asked to provide a video file for the ACM DL for the conference proceedings.
Presentation Details
More information will be posted here about preparing for in-person and online presentation as it becomes available.
Please check out the tab Online Authors’ Corner and Instructions.
Resources
Selecting a Track
There are many resources for writing high quality papers for submission to the SIGCSE Technical Symposium. We encourage authors to read and evaluate papers from a prior SIGCSE Technical Symposium, especially those designated as best papers, which were selected both due to content and high quality reporting.
Here are the best papers from SIGCSE TS 2023 as examples that showcase the difference between the three paper tracks.
Computing Education Research (CER)
-
Geoffrey L. Herman, Shan Huang, Peter A. Peterson, Linda Oliva, Enis Golaszewski, and Alan T. Sherman. 2023. Psychometric Evaluation of the Cybersecurity Curriculum Assessment. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 228–234. https://dl.acm.org/doi/10.1145/3545945.3569762
-
Rachel Harred, Tiffany Barnes, Susan R. Fisk, Bita Akram, Thomas W. Price, and Spencer Yoder. 2023. Do Intentions to Persist Predict Short-Term Computing Course Enrollments: A Scale Development, Validation, and Reliability Analysis. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 1062–1068. https://dl.acm.org/doi/10.1145/3545945.3569875
-
Eric J. Mayhew and Elizabeth Patitsas. 2023. Critical Pedagogy in Practice in the Computing Classroom. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 1076–1082. https://dl.acm.org/doi/10.1145/3545945.3569840
Experience Reports and Tools (ERT)
- Bailey Flanigan, Ananya A. Joshi, Sara McAllister, and Catalina Vajiac. 2023. CS-JEDI: Required DEI Education, by CS PhD Students, for CS PhD Students. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 87–93. https://dl.acm.org/doi/10.1145/3545945.3569733
- Gloria Ashiya Katuka, Yvonika Auguste, Yukyeong Song, Xiaoyi Tian, Amit Kumar, Mehmet Celepkolu, Kristy Elizabeth Boyer, Joanne Barrett, Maya Israel, and Tom McKlin. 2023. A Summer Camp Experience to Engage Middle School Learners in AI through Conversational App Development. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 813–819. https://dl.acm.org/doi/10.1145/3545945.3569864
- Lisa Zhang, Bogdan Simion, Michael Kaler, Amna Liaqat, Daniel Dick, Andi Bergen, Michael Miljanovic, and Andrew Petersen. 2023. Embedding and Scaling Writing Instruction Across First- and Second-Year Computer Science Courses. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 610–616. https://dl.acm.org/doi/10.1145/3545945.3569729
Position and Curricula Initiative (PCI)
- Brett A. Becker, Paul Denny, James Finnie-Ansley, Andrew Luxton-Reilly, James Prather, and Eddie Antonio Santos. 2023. Programming Is Hard - Or at Least It Used to Be: Educational Opportunities and Challenges of AI Code Generation. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 500–506. https://dl.acm.org/doi/10.1145/3545945.3569759
- Muwei Zheng, Nathan Swearingen, Steven Mills, Croix Gyurek, Matt Bishop, and Xukai Zou. 2023. Case Study: Mapping an E-Voting Based Curriculum to CSEC2017. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 514–520. https://dl.acm.org/doi/10.1145/3545945.3569811
Authors will also likely find the paper review guidelines beneficial for identifying how reviewers will assess papers for each track. Below, we list additional resources that you may find useful as you write your papers, especially computing education research papers.
- Writing a research question (csedresearch.org)
- Reporting Tips (csedresearch.org)
- Checklist for Research Articles (csedresearch.org)
- Evaluation Instruments (csedresearch.org)
- What’s the difference between a research paper and an experience report? (Amy Ko)
Language Editing Assistance
ACM has partnered with International Science Editing (ISE) to provide language editing services to ACM authors. ISE offers a comprehensive range of services for authors including standard and premium English language editing, as well as illustration and translation services. Editing services are at author expense and do not guarantee publication of a manuscript.
Instructions for Reviewers
This page captures the reviewing policies of the papers tracks at SIGCSE TS. Please email the Program Chairs at program@sigcse2024.sigcse.org with comments or questions.
There are three different paper types at SIGCSE TS : Computing Education Research (CER), Experience Reports and Tools (ERT), and Position and Curricula Initiative (PCI). When authors submit a paper, they have to select to submit to one of the three different types of papers.
- Submission System
- Roles in the Review Process
- Timeline
- Paper Reviewing Guidelines (CER, ERT, and PCI)
- Review Process Steps
- Conflicts of Interest
- Recalcitrant Reviewers
Submission System
The review process for SIGCSE TS 2024 will be done using the EasyChair submission system (https://easychair.org/my/conference?conf=sigcsets2024) . Reviewers will be invited to join/login into EasyChair, update their profile, and select 3-5 topics that they are most qualified to review. To do so, reviewers select SIGCSE TS 2024 > Conference > My topics from the menu and select at most 5 topics. More topics make it harder for the EasyChair system to make a good set of matches. Reviewers also identify their Conflicts of Interest by selecting SIGCSE TS 2024 > Conference > My Conflicts.
Roles in the Review Process
- Reviewers write reviews of their assigned submissions, evaluating them against the review criteria.
- Associate Program Chairs (APCs) write meta-review for their assigned submissions and provide a recommendation (accept/reject) and feedback to the Program Chairs.
- Program Chairs make the final decisions on the program based on recommendations from the APCs (for papers) and from track chairs (for other tracks).
SIGCSE TS has three Program Chairs, each of whom serves a two-year term. Nominations for Program Chairs are solicited by the SIGCSE TS steering committee, which makes recommendations to the SIGCSE Board. Program Chairs are appointed by the SIGCSE board.
The Program Chairs invite and appoint the Reviewers and APCs. The number of submissions per Reviewer/APC depends on the number of volunteers and the size of the submissions pool.
The goals is for each paper submission to receive at least three reviews and a meta-review. All reviews are submitted through the submission system. In EasyChair, Reviewers are considered “Ordinary PC members” and APCs are considered “Senior PC members”.
Timeline
| Reviewing Phase | Start Date | End Date |
|---|---|---|
| Bidding | Saturday, 12 August 2023 | Wednesday, 16 August 2023 |
| Reviewing | Saturday, 19 August 2023 | Wednesday, 6 September 2023 |
| Discussion & Recommendations | Thursday, 7 September 2023 | Saturday, 16 September 2023 |
Note: Associate Program Chair (APC) Recommendation and Meta-Review Deadline: 11:59 p.m. Friday, 16 September 2023 anywhere on earth (AOE)
Paper Reviewing Guidelines (CER, ERT, and PCI)
There are three different paper types at SIGCSE TS : Computing Education Research (CER), Experience Reports and Tools (ERT), and Position and Curricula Initiative (PCI). Reviewers are assigned to a specific paper track (e.g. a reviewer in the CER track will only be assigned to review papers in that track). This is to avoid confusion and for reviewers to get familiar with the guidelines for their specific paper track.
All papers will be considered relative to criteria for motivation, use of prior/related work, approach, evidence, contribution/impact, and presentation. Each track has guidance about how reviewers should consider these criteria relative to the goal of the track, and each paper must be evaluated using the criteria for the track to which it is submitted. A paper will not be moved between the three paper tracks.
The following table illustrates how to interpret the review criteria for each of the three tracks of papers. Please refer to this table to help better understand the emphases or characteristics of the track for which you will be reviewing. For convenience, you may also download a PDF copy of the paper review criteria.
| Criteria | Computing Education Research (CER) | Experience Reports & Tools (ERT) | Position & Curricula Initiative (PCI) |
|---|---|---|---|
| Motivation
Evaluate the submissions clarity of purpose and alignment with the scope of the SIGCSE TS. |
|
|
|
| Prior and Related Work
Evaluate the use of prior literature to situate the work, highlight its novelty, and interpret its results. |
|
|
|
| Approach
Evaluate the transparency and soundness of the approach used in the submission relative to its goals. |
|
|
|
| Evidence
Evaluate the extent to which the submission provides adequate evidence to support its claims. |
|
|
|
| Contribution & Impact
Evaluate the overall contribution to computing education made by this submission. |
|
|
|
| Presentation
Evaluate the writing quality with respect to expectations for publication, allowing for only minor revisions prior to final submission. |
|
|
|
Review Process Steps
Step 1: Authors submit Abstracts of Papers
Authors submit a title and abstract one week prior to the full paper deadline. Authors are allowed to revise their title, abstract, and other information before the full paper submission deadline.
Step 2: Reviewers and APCs Bid for Papers
Reviewers and APCs select topics they feel most qualified to review. This helps the system prioritize papers.
Reviewers and APCs are then asked to select a set of papers for which they have sufficient expertise (we call this “bidding”). The Program Chairs assign papers based on these bids. The purpose of bidding is NOT to express interest in papers you want to read. It is to express your expertise and eligibility for fairly evaluating the work. These are subtly but importantly different purposes. We ask reviewers and APCs to select more papers than they plan to review so that we can best ensure that every paper has at least three reviewers.
- Make sure to specify all of your Conflicts of Interest.
- Bid on all of the papers you believe you have sufficient expertise to review.
- Do NOT bid on papers about topics, techniques, or methods that you oppose.
Step 3: Authors submit Full Papers
Submissions of the full papers are due one week after the abstracts are due. As indicated in the Instructions for Authors, submissions are supposed to be sufficiently anonymous so that the reviewer cannot determine the identity or affiliation of the authors. The main purpose of the anonymous reviewing process is to reduce the influence of potential (positive or negative) biases on reviewers’ assessments. You should be able to review the work without knowing the authors or their affiliations. Do not try to find out the identity of authors. When in doubt, please contact the Program Chairs.
Step 4: Program Chairs Decide on Desk Rejects
The Program Chairs will quickly review each paper submission to determine whether it violates anonymization requirements, length restrictions, or plagiarism policies. Authors of desk-rejected papers will be notified immediately. The Program Chairs may not catch every issue. If you see something during the review process that you believe should be desk rejected, contact the Program Chairs at program@sigcse2024.sigcse.org before you write a review. The Program Chairs will make the final judgment about whether something is a violation, and give you guidance on whether and if so how to write a review. Note that Program Chairs with conflicts of interest are excluded from deciding on desk-rejected papers, leaving the decision to the other Program Chairs.
Step 5: Program Chairs Assign Reviewers and APCs
Based on the bids and their judgment, the Program Chairs will collaboratively assign at least three Reviewers and one APC (meta-reviewer) for each paper submission. The Program Chairs will be advised by the submission system assignment algorithm, which depends on all bids being high quality. For the reviewer assignments to be fair and good, the reviewer bids should only be based on expertise and eligibility. Interest alone is not sufficient for bidding to review a paper. Reviewing assignments can only be made by a Program Chair without a conflict of interest.
Step 6a: Reviewers Review Papers
Assigned Reviewers submit their anonymous reviews by the review deadline, reviewing each of their assigned submissions against the Paper Reviewing Guidelines (CER, ERT, and PCI). We strongly recommend that you prepare your rationale in a separate document; EasyChair has been known to time out.
Note that Reviewers must NOT include accept or reject decisions in their review text. (They will indicate accept/reject recommendations separately.)
Due to the internal and external (publication) deadlines, we generally cannot give reviewers or APCs extensions. Note that reviewers, meta-reviewers, and Program Chairs with conflicts cannot see any of the reviews of the papers for which they have conflicts of interest during this process.
Step 6b: APCs and Program Chairs Monitor Review Progress
APCs and Program Chairs periodically check in to ensure that progress is being made. If needed, reminders are emailed to the reviewers with the expectations and timelines. If needed, the Program Chairs recruit emergency reviewers if any of the submissions do not have a sufficient number of reviews, if there is lots of variability in the reviews, or if an expert review is needed.
Step 7: Discussion between Reviewers and APCs
The discussion period provides the opportunity for the Reviewers and the APCs to discuss the reviews and reach an agreement on the quality of the submission relative to the expectations for the track to which it was submitted. The APCs are expected to take leadership role and moderate the discussion. Reviewers are expected to engage in the discussion when prompted by other Reviewers and/or by the APCs by using the Comments feature of EasyChair.
During the discussion period, Reviewers are able to revise their reviews but are NOT required to do so. It is important that at no point Reviewers feel forced to change their reviews, scores, or viewpoints in this process. The APC can disagree with the reviewers and communicate this to the Program Chairs if needed. Everyone is asked to do the following:
- Read all the reviews of all papers assigned (and re-read your own reviews).
- Engage in a discussion about sources of disagreement.
- Use the Paper Reviewing Guidelines (CER, ERT and PCI) to guide your discussions.
- Be polite, friendly, and constructive at all times.
- Be responsive and react as soon as new information comes in.
- Remain open to other reviewers shifting your judgments.
- Explicitly state any clarifying questions that could change your evaluation of the paper
At the end of the discussion period, the APCs should have enough feedback so that they can make a recommendation for acceptance or rejection to the Program Chairs. This recommendation should be based on their own reading of the reviews and discussion, not simply on the overall score.
Step 8: APCs Write Meta-Reviews
Toward the end of the discussion period, APCs use the reviews, the discussion, and their own evaluation of the submission to write a meta-review and a recommendation for the Program Chairs. A meta-review should summarize the key strengths and weaknesses of the submission, in light of the Paper Reviewing Guidelines (CER, ERT, and PCI) and explain how these led to their recommendation decision. The summary and explanation should help the authors in revising their work where appropriate. The meta-review must constructively summarize all reviews and the discussion as well as summarize any open questions and doubts. A generic meta-review (“After long discussion, the reviewers decided that the paper is not up to standards, and therefore rejected the paper”) is not sufficient.
APCs do not include their recommendation for acceptance or rejection of a paper in their meta-review because they only see a small portion of the submitted papers. Instead, the APCs are asked to make a recommendation of accept or reject to the Program Chairs via the submission system. If however, the Reviewers had differing views and a consensus could not be reached, then the APC captures the essence of all reviews and leaves their recommendation as neutral, and the submission is then further discussed by the Program Chairs.
Recommendations should NOT be based only on scores. For example, an APC may decide to recommend rejection for a paper with three weak accepts, but recommend acceptance for a paper with two accepts and one strong reject (or vice versa)
Step 9: Program Chairs Make Decisions & Notify Authors
Before announcing decisions, the Program Chairs go through all the submissions and read all the reviews and meta-reviews to ensure clarity and consistency with the review process and its criteria as possible. This is done via synchronous meetings of the Program Chairs. APCs are consulted if needed. The Chairs make decisions based on recommendations and their own expertise as well as a desire to provide an appropriately varied program.
The Program Chairs then notify all authors of the decisions about their papers via the submission system.
Step 10: Evaluation
The Evaluation Chairs send out surveys to authors, reviewers, and APCs. Please take the time to respond to these surveys, as they inform processes and policies for future SIGCSE Technical Symposia.
The Program Chairs also request feedback from the APCs on the quality of reviews as a metric to be used for future invitations to review for the SIGCSE Technical Symposium.
New for 2024: We will do our best to identify a small set of exceptional reviewers who will receive reviewing awards at the symposium.
Conflicts of Interest
SIGCSE TS takes conflicts of interest, both real and perceived, quite seriously. The conference adheres to the ACM conflict of interest policy (https://www.acm.org/publications/policies/conflict-of-interest) as well as the SIGCSE conflict of interest policy (https://sigcse.org/policies/COI.html). These state that a paper submitted to the SIGCSE TS is a conflict of interest for an individual if at least one of the following is true:
- The individual is a co-author of the paper
- A student of the individual is a co-author of the paper
- The individual identifies the paper as a conflict of interest, i.e., that the individual does not believe that they can provide an impartial evaluation of the paper.
The following policies apply to conference organizers:
- The Program Chairs are not allowed to submit to any track.
- The chairs of any track are not allowed to submit to that specific track.
- All other conference organizers are allowed to submit to any track.
- All reviewers (PC members) and meta-reviewers (APC members) are allowed to submit to any track.
No reviewer, meta-reviewer, or chair with a conflict of interest in the paper will be included in any evaluation, discussion, or decision about the paper. It is the responsibility of the reviewers, meta-reviewers, and chairs to declare their conflicts of interest throughout the process. The corresponding actions are outlined below for each relevant step of the reviewing process. It is the responsibility of the chairs to ensure that no reviewer or meta-reviewer is assigned a role in the review process for any paper for which they have a conflict of interest.
Recalcitrant Reviewers
Reviewers who don’t submit reviews, have reviews with limited constructive feedback, do not engage effectively in the discussion phase, or submit inappropriate reviews will be removed from the reviewer list (as per SIGCSE policy). Recalcitrant reviewers will be informed of their removal from the reviewer list. Reviewers with repeated offenses (two within a three-year period) will be removed from SIGCSE reviewing for three years.
Review Forms
The approximate text from the review form follows. We anticipate making some changes to CER to provide greater guidance on dealing with qualitative and mixed-methods papers.
Note that not all reviewer responses are available to authors.
Common Introductory Fields
Summary: Please provide a brief summary of the submission, its audience, and its main point(s), with respect to the review criteria of this track. Refer to the Table on the SIGCSE TS 2024 website (i.e., Instructions for Reviewers) to familiarize yourself with the review criteria for the appropriate track: (1) Computing Education Research, (2) Experience Reports and Tools, and (3) Position and Curricula Initiatives.
Familiarity: Rate your personal familiarity with the topic area of this submission in relation to your research or practical experience.
- None - I have never reviewed or written a paper or otherwise have experience in this area
- Low - I have read papers or otherwise have slight experience in this area
- Medium - I have reviewed papers or otherwise have some experience in this area
- High - I have written and reviewed papers or otherwise have moderate experience in this area
- Expert - I have written and reviewed many papers or otherwise have extensive experience in this area
Computing Education Research
Motivation (CER): Evaluate the submission’s clarity of purpose and alignment with the scope of the SIGCSE TS.
- The submission provides a clear motivation for the work.
- The submission states a set of clear Research Questions or Specific Aims/Goals.
Prior and Related Work (CER): Evaluate the use of prior literature to situate the work, highlight its novelty, and interpret its results.
- Discussion of prior and related work (e.g., theories, recent empirical findings, curricular trends) to contextualize and motivate the research is adequate.
- The relationship between prior work and the current study is clearly stated.
- The work leverages theory where appropriate.
Approach (CER): Evaluate the transparency and soundness of the approach used in the submission relative to its goals.
- Study methods and data collection processes are transparent and clearly described.
- The methodology described is a valid/sound way to answer the research questions posed or address the aims of the study identified by the authors.
- The submission provides enough detail to support replication of the methods.
Evidence (CER): Evaluate the extent to which the submission provides adequate evidence to support its claims.
- The analysis & results are clearly presented and aligned with the research questions/goals.
- Qualitative or quantitative data is interpreted appropriately.
- Missing or noisy data is addressed.
- Claims are well supported by the data presented.
- The threats to validity and/or study limitations are clearly stated.
Contribution & Impact (CER): Evaluate the overall contribution to computing education made by this submission.
- All CER papers should advance our knowledge of computing education.
- Quantitative research should discuss generalizability or transferability of findings beyond the original context.
- Qualitative research should add deeper understanding about a specific context or problem.
- For novel projects, the contribution beyond prior work is explained.
- For replications, the contribution includes a discussion on the implications of the new results–even if null or negative–when compared to prior work.
Presentation (CER): Evaluate the writing quality with respect to expectations for publication, allowing for only minor revisions prior to final submission.
- The presentation (e.g., writing, grammar, graphs, diagrams) is clear.
- Overall flow and organization are appropriate.
Experience Reports and Tools
Motivation (ERT) Evaluate the submission’s clarity of purpose and alignment with the scope of the SIGCSE TS.
- The submission provides a clear motivation for the work.
- Objectives or goals of the experience report are clearly stated, with an emphasis on contextual factors that help readers interpret the work.
- ERT submissions need NOT be framed around a set of research questions or theoretical frameworks.
Prior and Related Work (ERT) Evaluate the use of prior literature to situate the work, highlight its novelty, and interpret its results.
- Discussion of prior and related work to contextualize and motivate the experience report is adequate.
- The relationship between prior work and the experience or tool is clearly stated.
Approach (ERT): Evaluate the transparency and soundness of the approach used in the submission relative to its goals.
- For tool-focused papers: Is the design of the tool appropriate for its stated goals? Is the context of its deployment clearly described?
- For experience report papers: Is the experience sufficiently described to understand how it was designed/executed and who the target learner populations were?
- For all papers: To what extent does the paper provide reasonable mechanisms of formative assessment about the experience or tool?
Evidence (ERT): Evaluate the extent to which the submission provides adequate evidence to support its claims.
- The submission provides rich reflection on what did or didn’t work, and why.
- Evidence presented in ERT papers is often descriptive or narrative in format, and may or may not be driven by explicit motivating questions.
- ERT papers may include small-scale studies, but they need not be statistically significant.
- Claims about the experience or tool are sufficiently scoped within the bounds of the evidence presented.
Contribution & Impact (ERT): Evaluate the overall contribution to computing education made by this submission.
- Why the submission is of interest to SIGCSE community is clearly explained.
- The work enables adoption by other practitioners.
- The work highlights the novelty of the experience or tool presented.
- The implications for future work/use are clearly stated.
Presentation (ERT): Evaluate the writing quality with respect to expectations for publication, allowing for only minor revisions prior to final submission.
- The presentation (e.g., writing, grammar, graphs, diagrams) is clear.
- Overall flow and organization are appropriate.
Position Papers and Curricular Initiatives
Motivation (PCI): Evaluate the submission’s clarity of purpose and alignment with the scope of the SIGCSE TS.
- The submission provides a clear motivation for the work.
- Objectives or goals of the position or curricula initiative are clearly stated, and speak to issues beyond a single course or experience.
- Submissions focused on curricula, programs, or degrees should describe the motivating context before the new initiative was undertaken.
- PCI papers may or may not ground the work in theory or research questions.
Prior and Related Work (PCI): Evaluate the use of prior literature to situate the work, highlight its novelty, and interpret its results.
- Discussion of prior and related work to contextualize and motivate the position or initiative is adequate.
- The relationship between prior work and the proposed initiative or position is clearly stated.
Approach (PCI): Evaluate the transparency and soundness of the approach used in the submission relative to its goals.
- The submission uses an appropriate mechanism to present and defend its stated position or curriculum proposal (this may include things like a scoping review, secondary data analysis, program evaluation, among others).
- As necessary, the approach used is clearly described.
- PCI papers leveraging a literature-driven argument need not necessarily use a systematic review format, though it may be appropriate for certain types of claims.
Evidence (PCI): Evaluate the extent to which the submission provides adequate evidence to support its claims.
- PCI papers need not present original data collection, but may leverage other forms of scholarly evidence to support the claims made.
- Evidence presented is sufficient for defending the position or curriculum initiative.
- Claims should be sufficiently scoped relative to the type of evidence presented.
Contribution & Impact (PCI) Evaluate the overall contribution to computing education made by this submission.
- The work presents a coherent argument about a computing education topic, including, but not limited to curriculum or program design, practical and social issues facing computing educators, and critiques of existing practices.
- The submission offers new insights about broader concerns to the computing education community or offers guidance for adoption of new curricular approaches.
Presentation (PCI): Evaluate the writing quality with respect to expectations for publication, allowing for only minor revisions prior to final submission.
- The presentation (e.g., writing, grammar, graphs, diagrams) is clear.
- Overall flow and organization are appropriate.
Common Text: Recommendation
Overall evaluation: Please provide a detailed justification that includes constructive feedback that summarizes the strengths & weaknesses of the submission and clarifies your scores. Both the score and the review text are required, but remember that the authors will not see the overall recommendation score (only your review text). You should NOT directly include your preference for acceptance or rejection in your review.
Instructions for APCs
Forthcoming.
For the time being, please review the Instructions for Reviewers, which capture most of the Associate Program Chairs’ responsibilities.
| Fri 18 Aug 2023 Submission Deadline |
| Fri 11 Aug 2023 Mandatory Abstract Deadline |
