-
The GPT Surprise: Offering Large Language Model Chat in a Massive Coding Class Reduced Engagement but Increased Adopters Exam Performances
Authors:
Allen Nie,
Yash Chandak,
Miroslav Suzara,
Malika Ali,
Juliette Woodrow,
Matt Peng,
Mehran Sahami,
Emma Brunskill,
Chris Piech
Abstract:
Large language models (LLMs) are quickly being adopted in a wide range of learning experiences, especially via ubiquitous and broadly accessible chat interfaces like ChatGPT and Copilot. This type of interface is readily available to students and teachers around the world, yet relatively little research has been done to assess the impact of such generic tools on student learning. Coding education…
▽ More
Large language models (LLMs) are quickly being adopted in a wide range of learning experiences, especially via ubiquitous and broadly accessible chat interfaces like ChatGPT and Copilot. This type of interface is readily available to students and teachers around the world, yet relatively little research has been done to assess the impact of such generic tools on student learning. Coding education is an interesting test case, both because LLMs have strong performance on coding tasks, and because LLM-powered support tools are rapidly becoming part of the workflow of professional software engineers. To help understand the impact of generic LLM use on coding education, we conducted a large-scale randomized control trial with 5,831 students from 146 countries in an online coding class in which we provided some students with access to a chat interface with GPT-4. We estimate positive benefits on exam performance for adopters, the students who used the tool, but over all students, the advertisement of GPT-4 led to a significant average decrease in exam participation. We observe similar decreases in other forms of course engagement. However, this decrease is modulated by the student's country of origin. Offering access to LLMs to students from low human development index countries increased their exam participation rate on average. Our results suggest there may be promising benefits to using LLMs in an introductory coding class, but also potential harms for engagement, which makes their longer term impact on student success unclear. Our work highlights the need for additional investigations to help understand the potential impact of future adoption and integration of LLMs into classrooms.
△ Less
Submitted 25 April, 2024;
originally announced July 2024.
-
Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts
Authors:
Tyler Angert,
Miroslav Ivan Suzara,
Jenny Han,
Christopher Lawrence Pondoc,
Hariharan Subramonyam
Abstract:
Creative coding tasks are often exploratory in nature. When producing digital artwork, artists usually begin with a high-level semantic construct such as a "stained glass filter" and programmatically implement it by varying code parameters such as shape, color, lines, and opacity to produce visually appealing results. Based on interviews with artists, it can be effortful to translate semantic cons…
▽ More
Creative coding tasks are often exploratory in nature. When producing digital artwork, artists usually begin with a high-level semantic construct such as a "stained glass filter" and programmatically implement it by varying code parameters such as shape, color, lines, and opacity to produce visually appealing results. Based on interviews with artists, it can be effortful to translate semantic constructs to program syntax, and current programming tools don't lend well to rapid creative exploration. To address these challenges, we introduce Spellburst, a large language model (LLM) powered creative-coding environment. Spellburst provides (1) a node-based interface that allows artists to create generative art and explore variations through branching and merging operations, (2) expressive prompt-based interactions to engage in semantic programming, and (3) dynamic prompt-driven interfaces and direct code editing to seamlessly switch between semantic and syntactic exploration. Our evaluation with artists demonstrates Spellburst's potential to enhance creative coding practices and inform the design of computational creativity tools that bridge semantic and syntactic spaces.
△ Less
Submitted 7 August, 2023;
originally announced August 2023.
-
Coding with Purpose: Learning AI in Rural California
Authors:
Stephanie Tena-Meza,
Miroslav Suzara,
AJ Alvero
Abstract:
We use an autoethnographic case study of a Latinx high school student from an agricultural community in California to highlight how AI is learned outside classrooms and how her personal background influenced her social-justice oriented applications of AI technologies. Applying the concept of learning pathways from the learning sciences, we argue that redesigning AI education to be more inclusive w…
▽ More
We use an autoethnographic case study of a Latinx high school student from an agricultural community in California to highlight how AI is learned outside classrooms and how her personal background influenced her social-justice oriented applications of AI technologies. Applying the concept of learning pathways from the learning sciences, we argue that redesigning AI education to be more inclusive with respect to socioeconomic status, ethnoracial identity, and gender is important in the development of computational projects that address social-injustice. We also learn about the role of institutions, power structures, and community as they relate to her journey of learning and applying AI. The future of AI, its potential to address issues of social injustice and limiting the negative consequences of its use, will depend on the participation and voice of students from the most vulnerable communities.
△ Less
Submitted 30 August, 2021;
originally announced August 2021.
