-
ORES-Inspect: A technology probe for machine learning audits on enwiki
Authors:
Zachary Levonian,
Lauren Hagen,
Lu Li,
Jada Lilleboe,
Solvejg Wastvedt,
Aaron Halfaker,
Loren Terveen
Abstract:
Auditing the machine learning (ML) models used on Wikipedia is important for ensuring that vandalism-detection processes remain fair and effective. However, conducting audits is challenging because stakeholders have diverse priorities and assembling evidence for a model's [in]efficacy is technically complex. We designed an interface to enable editors to learn about and audit the performance of the…
▽ More
Auditing the machine learning (ML) models used on Wikipedia is important for ensuring that vandalism-detection processes remain fair and effective. However, conducting audits is challenging because stakeholders have diverse priorities and assembling evidence for a model's [in]efficacy is technically complex. We designed an interface to enable editors to learn about and audit the performance of the ORES edit quality model. ORES-Inspect is an open-source web tool and a provocative technology probe for researching how editors think about auditing the many ML models used on Wikipedia. We describe the design of ORES-Inspect and our plans for further research with this system.
△ Less
Submitted 12 June, 2024;
originally announced June 2024.
-
A scoping review of using Large Language Models (LLMs) to investigate Electronic Health Records (EHRs)
Authors:
Lingyao Li,
Jiayan Zhou,
Zhenxiang Gao,
Wenyue Hua,
Lizhou Fan,
Huizi Yu,
Loni Hagen,
Yongfeng Zhang,
Themistocles L. Assimes,
Libby Hemphill,
Siyuan Ma
Abstract:
Electronic Health Records (EHRs) play an important role in the healthcare system. However, their complexity and vast volume pose significant challenges to data interpretation and analysis. Recent advancements in Artificial Intelligence (AI), particularly the development of Large Language Models (LLMs), open up new opportunities for researchers in this domain. Although prior studies have demonstrat…
▽ More
Electronic Health Records (EHRs) play an important role in the healthcare system. However, their complexity and vast volume pose significant challenges to data interpretation and analysis. Recent advancements in Artificial Intelligence (AI), particularly the development of Large Language Models (LLMs), open up new opportunities for researchers in this domain. Although prior studies have demonstrated their potential in language understanding and processing in the context of EHRs, a comprehensive scoping review is lacking. This study aims to bridge this research gap by conducting a scoping review based on 329 related papers collected from OpenAlex. We first performed a bibliometric analysis to examine paper trends, model applications, and collaboration networks. Next, we manually reviewed and categorized each paper into one of the seven identified topics: named entity recognition, information extraction, text similarity, text summarization, text classification, dialogue system, and diagnosis and prediction. For each topic, we discussed the unique capabilities of LLMs, such as their ability to understand context, capture semantic relations, and generate human-like text. Finally, we highlighted several implications for researchers from the perspectives of data resources, prompt engineering, fine-tuning, performance measures, and ethical concerns. In conclusion, this study provides valuable insights into the potential of LLMs to transform EHR research and discusses their applications and ethical considerations.
△ Less
Submitted 22 May, 2024; v1 submitted 5 May, 2024;
originally announced May 2024.
-
Recommendations for Planning Inclusive Astronomy Conferences
Authors:
Inclusive Astronomy 2 Local Organizing Committee,
:,
Brian Brooks,
Keira Brooks,
Lea Hagen,
Nimish Hathi,
Samantha Hoffman,
James Paranilam,
Laura Prichard
Abstract:
The Inclusive Astronomy (IA) conference series aims to create a safe space where community members can listen to the experiences of marginalized individuals in astronomy, discuss actions being taken to address inequities, and give recommendations to the community for how to improve diversity, equity, and inclusion in astronomy. The first IA was held in Nashville, TN, USA, 17-19 June, 2015. The Inc…
▽ More
The Inclusive Astronomy (IA) conference series aims to create a safe space where community members can listen to the experiences of marginalized individuals in astronomy, discuss actions being taken to address inequities, and give recommendations to the community for how to improve diversity, equity, and inclusion in astronomy. The first IA was held in Nashville, TN, USA, 17-19 June, 2015. The Inclusive Astronomy 2 (IA2) conference was held in Baltimore, MD, USA, 14-15 October, 2019. The Inclusive Astronomy 2 (IA2) Local Organizing Committee (LOC) has put together a comprehensive document of recommendations for planning future Inclusive Astronomy conferences based on feedback received and lessons learned. While these are specific to the IA series, many parts will be applicable to other conferences as well. Please find the recommendations and accompanying letter to the community here: https://outerspace.stsci.edu/display/IA2/LOC+Recommendations.
△ Less
Submitted 21 July, 2020;
originally announced July 2020.
