Blogs (4) >>

ChatGPT is a conversational AI platform that can produce code to solve problems when provided with a natural language prompt. Prior work on similar AI models has shown that they perform well on typical intro-level Computer Science problems. However, little is known about the performance of such tools on Data Science (DS) problems. In this work, we assess the performance of ChatGPT on assignments from three DS courses with varying difficulty levels. First, we apply the raw assignment prompts provided to the students and find that ChatGPT performs well on assignments with dataset(s) descriptions and progressive question prompts, which divide the programming requirements into sub-problems. Then, we perform prompt engineering on the assignments for which ChatGPT had low performance. We find that the following prompt engineering techniques significantly increased ChatGPT’s performance: breaking down abstract questions into steps, breaking down steps into multiple prompts, providing descriptions of the dataset(s), including algorithmic details, adding specific instructions to entice specific actions, and removing extraneous information. Finally, we discuss how our findings suggest potential changes to curriculum design of DS courses.

Thu 21 Mar

Displayed time zone: Pacific Time (US & Canada) change

15:45 - 17:00
Small Colleges and Beyond, LLMs and MorePapers at Meeting Rooms D137-138
Chair(s): Colleen Bamford County College of Morris
15:45
25m
Talk
Implications of ChatGPT for Data Science EducationGlobal
Papers
Yiyin Shen University of Wisconsin-Madison, Xinyi Ai University of California San Diego, Adalbert Gerald Soosai Raj University of California, San Diego, Rogers Jeffrey Leo John Independent Researcher, Meenakshi Syamkumar University of Wisconsin-Madison
DOI
16:10
25m
Talk
Playing with Matches: Adopting Gale--Shapley for Managing Student Enrollments Beyond CS2
Papers
Anna Rafferty Carleton College, David Liben-Nowell Carleton College, Dave Musicant Carleton College, Emy Farley Bowdoin, Allie Lyman Carleton College, Ann May Carleton College
DOI
16:35
25m
Talk
The Case for LLM Workshops: The Responsible Use of Large Language Models by Faculty at Small Liberal Arts Universities
Papers
Chris Bopp St. Bonaventure University, Anne Foerst St. Bonaventure University, Brian Kellogg St. Bonaventure University
DOI