-
Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system
Authors:
Sumit Asthana,
Sagih Hilleli,
Pengcheng He,
Aaron Halfaker
Abstract:
Meetings play a critical infrastructural role in the coordination of work. In recent years, due to shift to hybrid and remote work, more meetings are moving to online Computer Mediated Spaces. This has led to new problems (e.g. more time spent in less engaging meetings) and new opportunities (e.g. automated transcription/captioning and recap support). Recent advances in large language models (LLMs…
▽ More
Meetings play a critical infrastructural role in the coordination of work. In recent years, due to shift to hybrid and remote work, more meetings are moving to online Computer Mediated Spaces. This has led to new problems (e.g. more time spent in less engaging meetings) and new opportunities (e.g. automated transcription/captioning and recap support). Recent advances in large language models (LLMs) for dialog summarization have the potential to improve the experience of meetings by reducing individuals' meeting load and increasing the clarity and alignment of meeting outputs. Despite this potential, they face technological limitation due to long transcripts and inability to capture diverse recap needs based on user's context. To address these gaps, we design, implement and evaluate in-context a meeting recap system. We first conceptualize two salient recap representations -- important highlights, and a structured, hierarchical minutes view. We develop a system to operationalize the representations with dialogue summarization as its building blocks. Finally, we evaluate the effectiveness of the system with seven users in the context of their work meetings. Our findings show promise in using LLM-based dialogue summarization for meeting recap and the need for both representations in different contexts. However, we find that LLM-based recap still lacks an understanding of whats personally relevant to participants, can miss important details, and mis-attributions can be detrimental to group dynamics. We identify collaboration opportunities such as a shared recap document that a high quality recap enables. We report on implications for designing AI systems to partner with users to learn and improve from natural interactions to overcome the limitations related to personal relevance and summarization quality.
△ Less
Submitted 28 August, 2024; v1 submitted 28 July, 2023;
originally announced July 2023.
-
Modeling Inter-Dependence Between Time and Mark in Multivariate Temporal Point Processes
Authors:
Govind Waghmare,
Ankur Debnath,
Siddhartha Asthana,
Aakarsh Malhotra
Abstract:
Temporal Point Processes (TPP) are probabilistic generative frameworks. They model discrete event sequences localized in continuous time. Generally, real-life events reveal descriptive information, known as marks. Marked TPPs model time and marks of the event together for practical relevance. Conditioned on past events, marked TPPs aim to learn the joint distribution of the time and the mark of th…
▽ More
Temporal Point Processes (TPP) are probabilistic generative frameworks. They model discrete event sequences localized in continuous time. Generally, real-life events reveal descriptive information, known as marks. Marked TPPs model time and marks of the event together for practical relevance. Conditioned on past events, marked TPPs aim to learn the joint distribution of the time and the mark of the next event. For simplicity, conditionally independent TPP models assume time and marks are independent given event history. They factorize the conditional joint distribution of time and mark into the product of individual conditional distributions. This structural limitation in the design of TPP models hurt the predictive performance on entangled time and mark interactions. In this work, we model the conditional inter-dependence of time and mark to overcome the limitations of conditionally independent models. We construct a multivariate TPP conditioning the time distribution on the current event mark in addition to past events. Besides the conventional intensity-based models for conditional joint distribution, we also draw on flexible intensity-free TPP models from the literature. The proposed TPP models outperform conditionally independent and dependent models in standard prediction tasks. Our experimentation on various datasets with multiple evaluation metrics highlights the merit of the proposed approach.
△ Less
Submitted 27 October, 2022;
originally announced October 2022.
-
CARGO: AI-Guided Dependency Analysis for Migrating Monolithic Applications to Microservices Architecture
Authors:
Vikram Nitin,
Shubhi Asthana,
Baishakhi Ray,
Rahul Krishna
Abstract:
Microservices Architecture (MSA) has become a de-facto standard for designing cloud-native enterprise applications due to its efficient infrastructure setup, service availability, elastic scalability, dependability, and better security. Existing (monolithic) systems must be decomposed into microservices to harness these characteristics. Since manual decomposition of large scale applications can be…
▽ More
Microservices Architecture (MSA) has become a de-facto standard for designing cloud-native enterprise applications due to its efficient infrastructure setup, service availability, elastic scalability, dependability, and better security. Existing (monolithic) systems must be decomposed into microservices to harness these characteristics. Since manual decomposition of large scale applications can be laborious and error-prone, AI-based systems to detect decomposition strategies are gaining popularity. However, the usefulness of these approaches is limited by the expressiveness of the program representation and their inability to model the application's dependency on critical external resources such as databases. Consequently, partitioning recommendations offered by current tools result in architectures that result in (a) distributed monoliths, and/or (b) force the use of (often criticized) distributed transactions. This work attempts to overcome these challenges by introducing CARGO({short for [C]ontext-sensitive l[A]bel p[R]opa[G]ati[O]n})-a novel un-/semi-supervised partition refinement technique that uses a context- and flow-sensitive system dependency graph of the monolithic application to refine and thereby enrich the partitioning quality of the current state-of-the-art algorithms. CARGO was used to augment four state-of-the-art microservice partitioning techniques that were applied on five Java EE applications (including one industrial scale proprietary project). Experiments demostrate that CARGO can improve the partition quality of all modern microservice partitioning techniques. Further, CARGO substantially reduces distributed transactions and a real-world performance evaluation of a benchmark application (deployed under varying loads) shows that CARGO also lowers the overall the latency of the deployed microservice application by 11% and increases throughput by 120% on average.
△ Less
Submitted 6 October, 2022; v1 submitted 24 July, 2022;
originally announced July 2022.
-
Automatically Labeling Low Quality Content on Wikipedia by Leveraging Patterns in Editing Behaviors
Authors:
Sumit Asthana,
Sabrina Tobar Thommel,
Aaron Lee Halfaker,
Nikola Banovic
Abstract:
Wikipedia articles aim to be definitive sources of encyclopedic content. Yet, only 0.6% of Wikipedia articles have high quality according to its quality scale due to insufficient number of Wikipedia editors and enormous number of articles. Supervised Machine Learning (ML) quality improvement approaches that can automatically identify and fix content issues rely on manual labels of individual Wikip…
▽ More
Wikipedia articles aim to be definitive sources of encyclopedic content. Yet, only 0.6% of Wikipedia articles have high quality according to its quality scale due to insufficient number of Wikipedia editors and enormous number of articles. Supervised Machine Learning (ML) quality improvement approaches that can automatically identify and fix content issues rely on manual labels of individual Wikipedia sentence quality. However, current labeling approaches are tedious and produce noisy labels. Here, we propose an automated labeling approach that identifies the semantic category (e.g., adding citations, clarifications) of historic Wikipedia edits and uses the modified sentences prior to the edit as examples that require that semantic improvement. Highest-rated article sentences are examples that no longer need semantic improvements. We show that training existing sentence quality classification algorithms on our labels improves their performance compared to training them on existing labels. Our work shows that editing behaviors of Wikipedia editors provide better labels than labels generated by crowdworkers who lack the context to make judgments that the editors would agree with.
△ Less
Submitted 4 August, 2021;
originally announced August 2021.
-
HealthAdvisor: Recommendation System for Wearable Technologies enabling Proactive Health Monitoring
Authors:
Shubhi Asthana,
Ray Strong,
Aly Megahed
Abstract:
Proactive monitoring of one's health could avoid serious diseases as well as better maintain the individual's well-being. In today's IoT world, there has been numerous wearable technological devices to monitor/measure different health attributes. However, with that increasing number of attributes and wearables, it becomes unclear to the individual which ones they should be using. The aim of this p…
▽ More
Proactive monitoring of one's health could avoid serious diseases as well as better maintain the individual's well-being. In today's IoT world, there has been numerous wearable technological devices to monitor/measure different health attributes. However, with that increasing number of attributes and wearables, it becomes unclear to the individual which ones they should be using. The aim of this paper is to provide a recommendation engine for personalized recommended wearables for any given individual. The way the engine works is through first identifying the diseases that this person is at risk of, given his/her attributes and medical history. We built a machine learning classification model for this task. Second, these diseases are mapped to the attributes that need to be measured in order to monitor such diseases. Third, we map these measurements to the appropriate wearable technologies. This is done via a textual analytics model that we developed that uses available information of different wearables to map the aforementioned measurements to these wearables. The output can be used to recommend the wearables to individuals as well as provide a feedback to wearable developers for common measurements that do not have corresponding wearables today.
△ Less
Submitted 2 December, 2016;
originally announced December 2016.
-
Automated Document Indexing via Intelligent Hierarchical Clustering: A Novel Approach
Authors:
Rajendra Kumar Roul,
Shubham Rohan Asthana,
Sanjay Kumar Sahay
Abstract:
With the rising quantity of textual data available in electronic format, the need to organize it become a highly challenging task. In the present paper, we explore a document organization framework that exploits an intelligent hierarchical clustering algorithm to generate an index over a set of documents. The framework has been designed to be scalable and accurate even with large corpora. The adva…
▽ More
With the rising quantity of textual data available in electronic format, the need to organize it become a highly challenging task. In the present paper, we explore a document organization framework that exploits an intelligent hierarchical clustering algorithm to generate an index over a set of documents. The framework has been designed to be scalable and accurate even with large corpora. The advantage of the proposed algorithm lies in the need for minimal inputs, with much of the hierarchy attributes being decided in an automated manner using statistical methods. The use of topic modeling in a pre-processing stage ensures robustness to a range of variations in the input data. For experimental work 20-Newsgroups dataset has been used. The F- measure of the proposed approach has been compared with the traditional K-Means and K-Medoids clustering algorithms. Test results demonstrate the applicability, efficiency and effectiveness of our proposed approach. After extensive experimentation, we conclude that the framework shows promise for further research and specialized commercial applications.
△ Less
Submitted 1 April, 2015;
originally announced April 2015.
