-
Generating Situated Reflection Triggers about Alternative Solution Paths: A Case Study of Generative AI for Computer-Supported Collaborative Learning
Authors:
Atharva Naik,
Jessica Ruhan Yin,
Anusha Kamath,
Qianou Ma,
Sherry Tongshuang Wu,
Charles Murray,
Christopher Bogart,
Majd Sakr,
Carolyn P. Rose
Abstract:
An advantage of Large Language Models (LLMs) is their contextualization capability - providing different responses based on student inputs like solution strategy or prior discussion, to potentially better engage students than standard feedback. We present a design and evaluation of a proof-of-concept LLM application to offer students dynamic and contextualized feedback. Specifically, we augment an…
▽ More
An advantage of Large Language Models (LLMs) is their contextualization capability - providing different responses based on student inputs like solution strategy or prior discussion, to potentially better engage students than standard feedback. We present a design and evaluation of a proof-of-concept LLM application to offer students dynamic and contextualized feedback. Specifically, we augment an Online Programming Exercise bot for a college-level Cloud Computing course with ChatGPT, which offers students contextualized reflection triggers during a collaborative query optimization task in database design. We demonstrate that LLMs can be used to generate highly situated reflection triggers that incorporate details of the collaborative discussion happening in context. We discuss in depth the exploration of the design space of the triggers and their correspondence with the learning objectives as well as the impact on student learning in a pilot study with 34 students.
△ Less
Submitted 28 April, 2024;
originally announced April 2024.
-
A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education
Authors:
Jacob Doughty,
Zipiao Wan,
Anishka Bompelli,
Jubahed Qayum,
Taozhi Wang,
Juran Zhang,
Yujia Zheng,
Aidan Doyle,
Pragnya Sridhar,
Arav Agarwal,
Christopher Bogart,
Eric Keylor,
Can Kultur,
Jaromir Savelka,
Majd Sakr
Abstract:
There is a constant need for educators to develop and maintain effective up-to-date assessments. While there is a growing body of research in computing education on utilizing large language models (LLMs) in generation and engagement with coding exercises, the use of LLMs for generating programming MCQs has not been extensively explored. We analyzed the capability of GPT-4 to produce multiple-choic…
▽ More
There is a constant need for educators to develop and maintain effective up-to-date assessments. While there is a growing body of research in computing education on utilizing large language models (LLMs) in generation and engagement with coding exercises, the use of LLMs for generating programming MCQs has not been extensively explored. We analyzed the capability of GPT-4 to produce multiple-choice questions (MCQs) aligned with specific learning objectives (LOs) from Python programming classes in higher education. Specifically, we developed an LLM-powered (GPT-4) system for generation of MCQs from high-level course context and module-level LOs. We evaluated 651 LLM-generated and 449 human-crafted MCQs aligned to 246 LOs from 6 Python courses. We found that GPT-4 was capable of producing MCQs with clear language, a single correct choice, and high-quality distractors. We also observed that the generated MCQs appeared to be well-aligned with the LOs. Our findings can be leveraged by educators wishing to take advantage of the state-of-the-art generative models to support MCQ authoring efforts.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
From GPT-3 to GPT-4: On the Evolving Efficacy of LLMs to Answer Multiple-choice Questions for Programming Classes in Higher Education
Authors:
Jaromir Savelka,
Arav Agarwal,
Christopher Bogart,
Majd Sakr
Abstract:
We explore the evolving efficacy of three generative pre-trained transformer (GPT) models in generating answers for multiple-choice questions (MCQ) from introductory and intermediate Python programming courses in higher education. We focus on the differences in capabilities of the models prior to the release of ChatGPT (Nov '22), at the time of the release, and today (i.e., Aug '23). Recent studie…
▽ More
We explore the evolving efficacy of three generative pre-trained transformer (GPT) models in generating answers for multiple-choice questions (MCQ) from introductory and intermediate Python programming courses in higher education. We focus on the differences in capabilities of the models prior to the release of ChatGPT (Nov '22), at the time of the release, and today (i.e., Aug '23). Recent studies have established that the abilities of the OpenAI's GPT models to handle assessments originally designed for humans keep increasing as the newer more capable models are released. However, the qualitative differences in the capabilities and limitations of these models to reason about and/or analyze programming MCQs have been under-explored. We evaluated three OpenAI's GPT models on formative and summative MCQ assessments from three Python courses (530 questions) focusing on the qualitative differences in the evolving efficacy of the subsequent models. This study provides further evidence and insight into the trajectory of the current developments where there already exists a technology that can be utilized by students to collect passing scores, with no effort whatsoever, on what today counts as viable programming knowledge and skills assessments. This study could be leveraged by educators and institutions to better understand the recent technological developments in order to adapt the design of programming assessments as well as to fuel the necessary discussions into how assessments in future programming classes should be updated.
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Harnessing LLMs in Curricular Design: Using GPT-4 to Support Authoring of Learning Objectives
Authors:
Pragnya Sridhar,
Aidan Doyle,
Arav Agarwal,
Christopher Bogart,
Jaromir Savelka,
Majd Sakr
Abstract:
We evaluated the capability of a generative pre-trained transformer (GPT-4) to automatically generate high-quality learning objectives (LOs) in the context of a practically oriented university course on Artificial Intelligence. Discussions of opportunities (e.g., content generation, explanation) and risks (e.g., cheating) of this emerging technology in education have intensified, but to date there…
▽ More
We evaluated the capability of a generative pre-trained transformer (GPT-4) to automatically generate high-quality learning objectives (LOs) in the context of a practically oriented university course on Artificial Intelligence. Discussions of opportunities (e.g., content generation, explanation) and risks (e.g., cheating) of this emerging technology in education have intensified, but to date there has not been a study of the models' capabilities in supporting the course design and authoring of LOs. LOs articulate the knowledge and skills learners are intended to acquire by engaging with a course. To be effective, LOs must focus on what students are intended to achieve, focus on specific cognitive processes, and be measurable. Thus, authoring high-quality LOs is a challenging and time consuming (i.e., expensive) effort. We evaluated 127 LOs that were automatically generated based on a carefully crafted prompt (detailed guidelines on high-quality LOs authoring) submitted to GPT-4 for conceptual modules and projects of an AI Practitioner course. We analyzed the generated LOs if they follow certain best practices such as beginning with action verbs from Bloom's taxonomy in regards to the level of sophistication intended. Our analysis showed that the generated LOs are sensible, properly expressed (e.g., starting with an action verb), and that they largely operate at the appropriate level of Bloom's taxonomy, respecting the different nature of the conceptual modules (lower levels) and projects (higher levels). Our results can be leveraged by instructors and curricular designers wishing to take advantage of the state-of-the-art generative models to support their curricular and course design efforts.
△ Less
Submitted 30 June, 2023;
originally announced June 2023.
-
Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses
Authors:
Jaromir Savelka,
Arav Agarwal,
Marshall An,
Chris Bogart,
Majd Sakr
Abstract:
This paper studies recent developments in large language models' (LLM) abilities to pass assessments in introductory and intermediate Python programming courses at the postsecondary level. The emergence of ChatGPT resulted in heated debates of its potential uses (e.g., exercise generation, code explanation) as well as misuses in programming classes (e.g., cheating). Recent studies show that while…
▽ More
This paper studies recent developments in large language models' (LLM) abilities to pass assessments in introductory and intermediate Python programming courses at the postsecondary level. The emergence of ChatGPT resulted in heated debates of its potential uses (e.g., exercise generation, code explanation) as well as misuses in programming classes (e.g., cheating). Recent studies show that while the technology performs surprisingly well on diverse sets of assessment instruments employed in typical programming classes the performance is usually not sufficient to pass the courses. The release of GPT-4 largely emphasized notable improvements in the capabilities related to handling assessments originally designed for human test-takers. This study is the necessary analysis in the context of this ongoing transition towards mature generative AI systems. Specifically, we report the performance of GPT-4, comparing it to the previous generations of GPT models, on three Python courses with assessments ranging from simple multiple-choice questions (no code involved) to complex programming projects with code bases distributed into multiple files (599 exercises overall). Additionally, we analyze the assessments that were not handled well by GPT-4 to understand the current limitations of the model, as well as its capabilities to leverage feedback provided by an auto-grader. We found that the GPT models evolved from completely failing the typical programming class' assessments (the original GPT-3) to confidently passing the courses with no human involvement (GPT-4). While we identified certain limitations in GPT-4's handling of MCQs and coding exercises, the rate of improvement across the recent generations of GPT models strongly suggests their potential to handle almost any type of assessment widely used in higher education programming courses. These findings could be leveraged by educators and institutions to adapt the design of programming assessments as well as to fuel the necessary discussions into how programming classes should be updated to reflect the recent technological developments. This study provides evidence that programming instructors need to prepare for a world in which there is an easy-to-use widely accessible technology that can be utilized by learners to collect passing scores, with no effort whatsoever, on what today counts as viable programming knowledge and skills assessments.
△ Less
Submitted 15 June, 2023;
originally announced June 2023.
-
SocioEconomicMag Meets a Platform for SES-Diverse College Students: A Case Study
Authors:
Puja Agarwal,
Divya Prem,
Christopher Bogart,
Abrar Fallatah,
Aileen Abril Castro-Guzman,
Pannapat Chanpaisaeng,
Stella Doehring,
Margaret Burnett,
Anita Sarma
Abstract:
Emerging research shows that individual differences in how people use technology sometimes cluster by socioeconomic status (SES) and that when technology is not socioeconomically inclusive, low-SES individuals may abandon it. To understand how to improve technology's SES-inclusivity, we present a multi-phase case study on SocioEconomicMag (SESMag), an emerging inspection method for socio+economic…
▽ More
Emerging research shows that individual differences in how people use technology sometimes cluster by socioeconomic status (SES) and that when technology is not socioeconomically inclusive, low-SES individuals may abandon it. To understand how to improve technology's SES-inclusivity, we present a multi-phase case study on SocioEconomicMag (SESMag), an emerging inspection method for socio+economic inclusivity. In our 16-month case study, a software team developing a learning management platform used SESMag to evaluate and then to improve their platform's SES-inclusivity. The results showed that (1) the practitioners identified SES-inclusivity bugs in 76% of the features they evaluated; (2) these inclusivity bugs actually arise among low-SES college students; and (3) the SESMag process pointed ways towards fixing these bugs. Finally, (4) a user study with SES-diverse college students showed that the platform's SES-inclusivity eradicated 45-54% of the bugs; for some types of bugs, the bug instance eradication rate was 80% or higher.
△ Less
Submitted 10 April, 2023;
originally announced April 2023.
-
Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?
Authors:
Jaromir Savelka,
Arav Agarwal,
Christopher Bogart,
Yifan Song,
Majd Sakr
Abstract:
We evaluated the capability of generative pre-trained transformers (GPT), to pass assessments in introductory and intermediate Python programming courses at the postsecondary level. Discussions of potential uses (e.g., exercise generation, code explanation) and misuses (e.g., cheating) of this emerging technology in programming education have intensified, but to date there has not been a rigorous…
▽ More
We evaluated the capability of generative pre-trained transformers (GPT), to pass assessments in introductory and intermediate Python programming courses at the postsecondary level. Discussions of potential uses (e.g., exercise generation, code explanation) and misuses (e.g., cheating) of this emerging technology in programming education have intensified, but to date there has not been a rigorous analysis of the models' capabilities in the realistic context of a full-fledged programming course with diverse set of assessment instruments. We evaluated GPT on three Python courses that employ assessments ranging from simple multiple-choice questions (no code involved) to complex programming projects with code bases distributed into multiple files (599 exercises overall). Further, we studied if and how successfully GPT models leverage feedback provided by an auto-grader. We found that the current models are not capable of passing the full spectrum of assessments typically involved in a Python programming course (<70% on even entry-level modules). Yet, it is clear that a straightforward application of these easily accessible models could enable a learner to obtain a non-trivial portion of the overall available score (>55%) in introductory and intermediate courses alike. While the models exhibit remarkable capabilities, including correcting solutions based on auto-grader's feedback, some limitations exist (e.g., poor handling of exercises requiring complex chains of reasoning steps). These findings can be leveraged by instructors wishing to adapt their assessments so that GPT becomes a valuable assistant for a learner as opposed to an end-to-end solution.
△ Less
Submitted 16 March, 2023;
originally announced March 2023.
-
Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code
Authors:
Jaromir Savelka,
Arav Agarwal,
Christopher Bogart,
Majd Sakr
Abstract:
We analyzed effectiveness of three generative pre-trained transformer (GPT) models in answering multiple-choice question (MCQ) assessments, often involving short snippets of code, from introductory and intermediate programming courses at the postsecondary level. This emerging technology stirs countless discussions of its potential uses (e.g., exercise generation, code explanation) as well as misus…
▽ More
We analyzed effectiveness of three generative pre-trained transformer (GPT) models in answering multiple-choice question (MCQ) assessments, often involving short snippets of code, from introductory and intermediate programming courses at the postsecondary level. This emerging technology stirs countless discussions of its potential uses (e.g., exercise generation, code explanation) as well as misuses in programming education (e.g., cheating). However, the capabilities of GPT models and their limitations to reason about and/or analyze code in educational settings have been under-explored. We evaluated several OpenAI's GPT models on formative and summative MCQ assessments from three Python courses (530 questions). We found that MCQs containing code snippets are not answered as successfully as those that only contain natural language. While questions requiring to fill-in a blank in the code or completing a natural language statement about the snippet are handled rather successfully, MCQs that require analysis and/or reasoning about the code (e.g., what is true/false about the snippet, or what is its output) appear to be the most challenging. These findings can be leveraged by educators to adapt their instructional practices and assessments in programming courses, so that GPT becomes a valuable assistant for a learner as opposed to a source of confusion and/or potential hindrance in the learning process.
△ Less
Submitted 9 March, 2023;
originally announced March 2023.
-
Intersectionality Goes Analytical: Taming Combinatorial Explosion Through Type Abstraction
Authors:
Margaret Burnett,
Martin Erwig,
Abrar Fallatah,
Christopher Bogart,
Anita Sarma
Abstract:
HCI researchers' and practitioners' awareness of intersectionality has been expanding, producing knowledge, recommendations, and prototypes for supporting intersectional populations. However, doing intersectional HCI work is uniquely expensive: it leads to a combinatorial explosion of empirical work (expense 1), and little of the work on one intersectional population can be leveraged to serve anot…
▽ More
HCI researchers' and practitioners' awareness of intersectionality has been expanding, producing knowledge, recommendations, and prototypes for supporting intersectional populations. However, doing intersectional HCI work is uniquely expensive: it leads to a combinatorial explosion of empirical work (expense 1), and little of the work on one intersectional population can be leveraged to serve another (expense 2). In this paper, we explain how representations employed by certain analytical design methods correspond to type abstractions, and use that correspondence to identify a (de)compositional model in which a population's diverse identity properties can be joined and split. We formally prove the model's correctness, and show how it enables HCI designers to harness existing analytical HCI methods for use on new intersectional populations of interest. We illustrate through four design use-cases, how the model can reduce the amount of expense 1 and enable designers to leverage prior work to new intersectional populations, addressing expense 2.
△ Less
Submitted 25 January, 2022;
originally announced January 2022.
-
Comparing Example-Based Collaborative Reflection to Problem Solving Practice for Learning during Team-Based Software Engineering Projects
Authors:
Sreecharan Sankaranarayanan,
Siddharth Reddy Kandimalla,
Christopher Bogart,
R. Charles Murray,
Haokang An,
Michael Hilton,
Majd Sakr,
Carolyn Rosé
Abstract:
Contributing to the literature on aptitude-treatment interactions between worked examples and problem-solving, this paper addresses differential learning from the two approaches when students are positioned as domain experts learning new concepts. Our evaluation is situated in a team project that is part of an advanced software engineering course. In this course, students who possess foundational…
▽ More
Contributing to the literature on aptitude-treatment interactions between worked examples and problem-solving, this paper addresses differential learning from the two approaches when students are positioned as domain experts learning new concepts. Our evaluation is situated in a team project that is part of an advanced software engineering course. In this course, students who possess foundational domain knowledge but are learning new concepts engage alternatively in programming followed by worked example-based reflection. They are either allowed to finish programming or are curtailed after a pre-specified time to participate in a longer worked example-based reflection. We find significant pre- to post-test learning gains in both conditions. Then, we not only find significantly more learning when students participated in longer worked example-based reflections but also a significant performance improvement on a problem-solving transfer task. These findings suggest that domain experts learning new concepts benefit more from worked example-based reflections than from problem-solving.
△ Less
Submitted 1 July, 2021;
originally announced July 2021.
-
Tracing Vulnerable Code Lineage
Authors:
David Reid,
Kalvin Eng,
Chris Bogart,
Adam Tutko
Abstract:
This paper presents results from the MSR 2021 Hackathon. Our team investigates files/projects that contain known security vulnerabilities and how widespread they are throughout repositories in open source software. These security vulnerabilities can potentially be propagated through code reuse even when the vulnerability is fixed in different versions of the code. We utilize the World of Code infr…
▽ More
This paper presents results from the MSR 2021 Hackathon. Our team investigates files/projects that contain known security vulnerabilities and how widespread they are throughout repositories in open source software. These security vulnerabilities can potentially be propagated through code reuse even when the vulnerability is fixed in different versions of the code. We utilize the World of Code infrastructure to discover file-level duplication of code from a nearly complete collection of open source software. This paper describes a method and set of tools to find all open source projects that use known vulnerable files and any previous revisions of those files.
△ Less
Submitted 23 March, 2021;
originally announced March 2021.
-
Guiding Development Work Across a Software Ecosystem by Visualizing Usage Data
Authors:
Christopher Bogart,
James Howison,
James Herbsleb
Abstract:
Software is increasingly produced in the form of ecosystems, collections of interdependent components maintained by a distributed community. These ecosystems act as network organizations, not markets, and thus often lack actionable price-like signals about how the software is used and what impact it has. We introduce a tool, the Scientific Software Network Map, that collects and displays summarize…
▽ More
Software is increasingly produced in the form of ecosystems, collections of interdependent components maintained by a distributed community. These ecosystems act as network organizations, not markets, and thus often lack actionable price-like signals about how the software is used and what impact it has. We introduce a tool, the Scientific Software Network Map, that collects and displays summarized usage data tailored to the needs of actors in software ecosystems. We performed a contextualized walkthrough of the Map with producers and stewards in six scientific software ecosystems that use the R language. We found that they work to maximize diversity rather than quantity of uses, and to minimize coordination costs. We also found that summarized usage data would be useful for justifying ecosystem work to funding agencies; and we discovered a variety of more granular usage needs that would help in adding or maintaining features.
△ Less
Submitted 10 December, 2020;
originally announced December 2020.
-
World of Code: Enabling a Research Workflow for Mining and Analyzing the Universe of Open Source VCS data
Authors:
Yuxing Ma,
Tapajit Dey,
Chris Bogart,
Sadika Amreen,
Marat Valiev,
Adam Tutko,
David Kennard,
Russell Zaretzki,
Audris Mockus
Abstract:
Open source software (OSS) is essential for modern society and, while substantial research has been done on individual (typically central) projects, only a limited understanding of the periphery of the entire OSS ecosystem exists. For example, how are the tens of millions of projects in the periphery interconnected through. technical dependencies, code sharing, or knowledge flow? To answer such qu…
▽ More
Open source software (OSS) is essential for modern society and, while substantial research has been done on individual (typically central) projects, only a limited understanding of the periphery of the entire OSS ecosystem exists. For example, how are the tens of millions of projects in the periphery interconnected through. technical dependencies, code sharing, or knowledge flow? To answer such questions we: a) create a very large and frequently updated collection of version control data in the entire FLOSS ecosystems named World of Code (WoC), that can completely cross-reference authors, projects, commits, blobs, dependencies, and history of the FLOSS ecosystems and b) provide capabilities to efficiently correct, augment, query, and analyze that data. Our current WoC implementation is capable of being updated on a monthly basis and contains over 18B Git objects. To evaluate its research potential and to create vignettes for its usage, we employ WoC in conducting several research tasks. In particular, we find that it is capable of supporting trend evaluation, ecosystem measurement, and the determination of package usage. We expect WoC to spur investigation into global properties of OSS development leading to increased resiliency of the entire OSS ecosystem. Our infrastructure facilitates the discovery of key technical dependencies, code flow, and social networks that provide the basis to determine the structure and evolution of the relationships that drive FLOSS activities and innovation.
△ Less
Submitted 30 October, 2020;
originally announced October 2020.
-
ALFAA: Active Learning Fingerprint Based Anti-Aliasing for Correcting Developer Identity Errors in Version Control Data
Authors:
Sadika Amreen,
Audris Mockus,
Chris Bogart,
Yuxia Zhang,
Russell Zaretzki
Abstract:
Graphs of developer networks are important for software engineering research and practice. For these graphs to realistically represent the networks, accurate developer identities are imperative. We aim to identify developer identity errors from open source software repositories in VCS, investigate the nature of these errors, design corrective algorithms, and estimate the impact of the errors on ne…
▽ More
Graphs of developer networks are important for software engineering research and practice. For these graphs to realistically represent the networks, accurate developer identities are imperative. We aim to identify developer identity errors from open source software repositories in VCS, investigate the nature of these errors, design corrective algorithms, and estimate the impact of the errors on networks inferred from this data. We investigate these questions using over 1B Git commits with over 23M recorded author identities. By inspecting the author strings that occur most frequently, we group identity errors into categories. We then augment the author strings with 3 behavioral fingerprints: time-zone frequencies, the set of files modified, and a vector embedding of the commit messages. We create a manually validated set of identities for a subset of OpenStack developers using an active learning approach and use it to fit supervised learning models to predict the identities for the remaining author strings in OpenStack. We compare these predictions with a commercial effort and a leading research method. Finally, we compare network measures for file-induced author networks based on corrected and raw data. We find commits done from different environments, misspellings, organizational IDs, default values, and anonymous IDs to be the major sources of errors. We also find supervised learning methods to reduce errors by several times in comparison to existing methods and the active learning approach to be an effective way to create validated datasets and that correction of developer identity has a large impact on the inference of the social network. We believe that our proposed Active Learning Fingerprint Based Anti-Aliasing (ALFAA) approach will expedite research progress in the software engineering domain for applications that depend upon graphs of developers or other social networks.
△ Less
Submitted 10 January, 2019;
originally announced January 2019.
-
Debugging Framework Applications: Benefits and Challenges
Authors:
Zack Coker,
David Gray Widder,
Claire Le Goues,
Christopher Bogart,
Joshua Sunshine
Abstract:
Aspects of frameworks, such as inversion of control and the structure of framework applications, require developers to adjust their debugging strategies as compared to debugging sequential programs. However, the benefits and challenges of framework debugging are not fully understood, and gaining this knowledge could provide guidance in debugging strategies and framework tool design. To gain insigh…
▽ More
Aspects of frameworks, such as inversion of control and the structure of framework applications, require developers to adjust their debugging strategies as compared to debugging sequential programs. However, the benefits and challenges of framework debugging are not fully understood, and gaining this knowledge could provide guidance in debugging strategies and framework tool design. To gain insight into the process developers use to fix problems in framework applications, we performed two human studies investigating how developers fix applications that use a framework API incorrectly. These studies focused on the Android Fragment class and the ROS framework. We analyzed the results of the studies using a mixed-methods approach, consisting of techniques from grounded theory, qualitative content analysis, and case studies. From our analysis, we produced a theory of the benefits and challenges of framework debugging. This theory states that developers find inversion of control challenging when debugging but find the structure of framework applications helpful. This theory could be used to guide strategies for debugging framework applications and framework tool designs.
△ Less
Submitted 16 January, 2018;
originally announced January 2018.
-
Linguistic Markers of Influence in Informal Interactions
Authors:
Shrimai Prabhumoye,
Samridhi Choudhary,
Evangelia Spiliopoulou,
Christopher Bogart,
Carolyn Penstein Rose,
Alan W Black
Abstract:
There has been a long standing interest in understanding `Social Influence' both in Social Sciences and in Computational Linguistics. In this paper, we present a novel approach to study and measure interpersonal influence in daily interactions. Motivated by the basic principles of influence, we attempt to identify indicative linguistic features of the posts in an online knitting community. We pres…
▽ More
There has been a long standing interest in understanding `Social Influence' both in Social Sciences and in Computational Linguistics. In this paper, we present a novel approach to study and measure interpersonal influence in daily interactions. Motivated by the basic principles of influence, we attempt to identify indicative linguistic features of the posts in an online knitting community. We present the scheme used to operationalize and label the posts with indicator features. Experiments with the identified features show an improvement in the classification accuracy of influence by 3.15%. Our results illustrate the important correlation between the characteristics of the language and its potential to influence others.
△ Less
Submitted 14 July, 2017;
originally announced July 2017.