-
GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applications
Authors:
Shishir G. Patil,
Tianjun Zhang,
Vivian Fang,
Noppapon C.,
Roy Huang,
Aaron Hao,
Martin Casado,
Joseph E. Gonzalez,
Raluca Ada Popa,
Ion Stoica
Abstract:
Large Language Models (LLMs) are evolving beyond their classical role of providing information within dialogue systems to actively engaging with tools and performing actions on real-world applications and services. Today, humans verify the correctness and appropriateness of the LLM-generated outputs (e.g., code, functions, or actions) before putting them into real-world execution. This poses signi…
▽ More
Large Language Models (LLMs) are evolving beyond their classical role of providing information within dialogue systems to actively engaging with tools and performing actions on real-world applications and services. Today, humans verify the correctness and appropriateness of the LLM-generated outputs (e.g., code, functions, or actions) before putting them into real-world execution. This poses significant challenges as code comprehension is well known to be notoriously difficult. In this paper, we study how humans can efficiently collaborate with, delegate to, and supervise autonomous LLMs in the future. We argue that in many cases, "post-facto validation" - verifying the correctness of a proposed action after seeing the output - is much easier than the aforementioned "pre-facto validation" setting. The core concept behind enabling a post-facto validation system is the integration of an intuitive undo feature, and establishing a damage confinement for the LLM-generated actions as effective strategies to mitigate the associated risks. Using this, a human can now either revert the effect of an LLM-generated output or be confident that the potential risk is bounded. We believe this is critical to unlock the potential for LLM agents to interact with applications and services with limited (post-facto) human involvement. We describe the design and implementation of our open-source runtime for executing LLM actions, Gorilla Execution Engine (GoEX), and present open research questions towards realizing the goal of LLMs and applications interacting with each other with minimal human supervision. We release GoEX at https://github.com/ShishirPatil/gorilla/.
△ Less
Submitted 10 April, 2024;
originally announced April 2024.
-
CDAD-Net: Bridging Domain Gaps in Generalized Category Discovery
Authors:
Sai Bhargav Rongali,
Sarthak Mehrotra,
Ankit Jha,
Mohamad Hassan N C,
Shirsha Bose,
Tanisha Gupta,
Mainak Singha,
Biplab Banerjee
Abstract:
In Generalized Category Discovery (GCD), we cluster unlabeled samples of known and novel classes, leveraging a training dataset of known classes. A salient challenge arises due to domain shifts between these datasets. To address this, we present a novel setting: Across Domain Generalized Category Discovery (AD-GCD) and bring forth CDAD-NET (Class Discoverer Across Domains) as a remedy. CDAD-NET is…
▽ More
In Generalized Category Discovery (GCD), we cluster unlabeled samples of known and novel classes, leveraging a training dataset of known classes. A salient challenge arises due to domain shifts between these datasets. To address this, we present a novel setting: Across Domain Generalized Category Discovery (AD-GCD) and bring forth CDAD-NET (Class Discoverer Across Domains) as a remedy. CDAD-NET is architected to synchronize potential known class samples across both the labeled (source) and unlabeled (target) datasets, while emphasizing the distinct categorization of the target data. To facilitate this, we propose an entropy-driven adversarial learning strategy that accounts for the distance distributions of target samples relative to source-domain class prototypes. Parallelly, the discriminative nature of the shared space is upheld through a fusion of three metric learning objectives. In the source domain, our focus is on refining the proximity between samples and their affiliated class prototypes, while in the target domain, we integrate a neighborhood-centric contrastive learning mechanism, enriched with an adept neighborsmining approach. To further accentuate the nuanced feature interrelation among semantically aligned images, we champion the concept of conditional image inpainting, underscoring the premise that semantically analogous images prove more efficacious to the task than their disjointed counterparts. Experimentally, CDAD-NET eclipses existing literature with a performance increment of 8-15% on three AD-GCD benchmarks we present.
△ Less
Submitted 8 April, 2024;
originally announced April 2024.
-
AI Based Waste classifier with Thermo-Rapid Composting
Authors:
Saswati kumari behera,
Aouthithiye Barathwaj SR Y,
Vasundhara L,
Saisudha G,
Haariharan N C
Abstract:
Waste management is a certainly a very complex and difficult process especially in very large cities. It needs immense man power and also uses up other resources such as electricity and fuel. This creates a need to use a novel method with help of latest technologies. Here in this article we present a new waste classification technique using Computer Vision (CV) and deep learning (DL). To further i…
▽ More
Waste management is a certainly a very complex and difficult process especially in very large cities. It needs immense man power and also uses up other resources such as electricity and fuel. This creates a need to use a novel method with help of latest technologies. Here in this article we present a new waste classification technique using Computer Vision (CV) and deep learning (DL). To further improve waste classification ability, support machine vectors (SVM) are used. We also decompose the degradable waste with help of rapid composting. In this article we have mainly worked on segregation of municipal solid waste (MSW). For this model, we use YOLOv3 (You Only Look Once) a computer vision-based algorithm popularly used to detect objects which is developed based on Convolution Neural Networks (CNNs) which is a machine learning (ML) based tool. They are extensively used to extract features from a data especially image-oriented data. In this article we propose a waste classification technique which will be faster and more efficient. And we decompose the biodegradable waste by Berkley Method of composting (BKC)
△ Less
Submitted 3 August, 2021;
originally announced August 2021.
-
AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages
Authors:
Anoop Kunchukuttan,
Divyanshu Kakwani,
Satish Golla,
Gokul N. C.,
Avik Bhattacharyya,
Mitesh M. Khapra,
Pratyush Kumar
Abstract:
We present the IndicNLP corpus, a large-scale, general-domain corpus containing 2.7 billion words for 10 Indian languages from two language families. We share pre-trained word embeddings trained on these corpora. We create news article category classification datasets for 9 languages to evaluate the embeddings. We show that the IndicNLP embeddings significantly outperform publicly available pre-tr…
▽ More
We present the IndicNLP corpus, a large-scale, general-domain corpus containing 2.7 billion words for 10 Indian languages from two language families. We share pre-trained word embeddings trained on these corpora. We create news article category classification datasets for 9 languages to evaluate the embeddings. We show that the IndicNLP embeddings significantly outperform publicly available pre-trained embedding on multiple evaluation tasks. We hope that the availability of the corpus will accelerate Indic NLP research. The resources are available at https://github.com/ai4bharat-indicnlp/indicnlp_corpus.
△ Less
Submitted 30 April, 2020;
originally announced May 2020.
-
Assessing Developer Beliefs: A Reply to "Perceptions, Expectations, and Challenges in Defect Prediction"
Authors:
Shrikanth N. C.,
Tim Menzies
Abstract:
It can be insightful to extend qualitative studies with a secondary quantitative analysis (where the former suggests insightful questions that the latter can answer). Documenting developer beliefs should be the start, not the end, of Software Engineering research. Once prevalent beliefs are found, they should be checked against real-world data. For example, this paper finds several notable discrep…
▽ More
It can be insightful to extend qualitative studies with a secondary quantitative analysis (where the former suggests insightful questions that the latter can answer). Documenting developer beliefs should be the start, not the end, of Software Engineering research. Once prevalent beliefs are found, they should be checked against real-world data. For example, this paper finds several notable discrepancies between empirical evidence and the developer beliefs documented in Wan et al.'s recent TSE paper "Perceptions, expectations, and challenges in defect prediction". By reporting these discrepancies we can stop developers (a) wasting time on inconsequential matters or (b) ignoring important effects. For the future, we would encourage more "extension studies" of prior qualitative results with quantitative empirical evidence.
△ Less
Submitted 11 April, 2019;
originally announced April 2019.
-
Trustworthiness in Enterprise Crowdsourcing: a Taxonomy & evidence from data
Authors:
Anurag Dwarakanath,
Shrikanth N. C.,
Kumar Abhinav,
Alex Kass
Abstract:
In this paper we study the trustworthiness of the crowd for crowdsourced software development. Through the study of literature from various domains, we present the risks that impact the trustworthiness in an enterprise context. We survey known techniques to mitigate these risks. We also analyze key metrics from multiple years of empirical data of actual crowdsourced software development tasks from…
▽ More
In this paper we study the trustworthiness of the crowd for crowdsourced software development. Through the study of literature from various domains, we present the risks that impact the trustworthiness in an enterprise context. We survey known techniques to mitigate these risks. We also analyze key metrics from multiple years of empirical data of actual crowdsourced software development tasks from two leading vendors. We present the metrics around untrustworthy behavior and the performance of certain mitigation techniques. Our study and results can serve as guidelines for crowdsourced enterprise software development.
△ Less
Submitted 25 September, 2018;
originally announced September 2018.
