Improvement in Program Repair Methods using Refactoring with GPT ModelsOnlineGlobalIn-Person
Teachers often utilize automatic program repair methods to provide feedback on submitted student code using model answer code. A state-of-the-art tool is Refactory, which achieves a high repair success rate and small patch size (less code repair) by refactoring code to expand the variety of correct code samples that can be referenced. However, Refactory has two major limitations. First, it cannot fix code with syntax errors. Second, it has difficulty fixing code when there are few correct submissions. Herein we propose a new method that combines Refactory and OpenAI’s GPT models to address these issues and conduct a performance measurement experiment. The experiment uses a dataset consisting of 5 programming assignment problems and almost 1,800 real-life incorrect Python program submissions from 361 students for an introductory programming course at a large public university. The proposed method improves the repair success rate by 1–21% when the set of correct code samples is sufficient and the patch size is smaller than Refactory alone in 16-45% of the cases. When there was no set of correct code samples at all (only the model answer code was used as a reference for repair), method improves the repair success rate by 1-43% and the patch size is smaller than Refactory alone in 42-68% of the cases.
Sat 23 MarDisplayed time zone: Pacific Time (US & Canada) change
10:45 - 12:00 | |||
10:45 25mTalk | A Self-Regulated Learning Framework using Generative AI and its Application in CS Educational Intervention DesignOnlineGlobalIn-Person Papers DOI | ||
11:10 25mTalk | Improvement in Program Repair Methods using Refactoring with GPT ModelsOnlineGlobalIn-Person Papers Ryosuke Ishizue NTT DATA Group Corporation / Waseda University, Kazunori Sakamoto WillBooster Inc. / Tokyo Online Unicersity / Waseda University, Hironori Washizaki Waseda University, Yoshiaki Fukazawa Waseda University DOI | ||
11:35 25mTalk | Using GPT-4 to Provide Tiered, Formative Code FeedbackOnlineIn-Person Papers DOI |