-
Natural Language Outlines for Code: Literate Programming in the LLM Era
Authors:
Kensen Shi,
Deniz Altınbüken,
Saswat Anand,
Mihai Christodorescu,
Katja Grünwedel,
Alexa Koenings,
Sai Naidu,
Anurag Pathak,
Marc Rasi,
Fredde Ribeiro,
Brandon Ruffin,
Siddhant Sanyam,
Maxim Tabachnyk,
Sara Toth,
Roy Tu,
Tobias Welp,
Pengcheng Yin,
Manzil Zaheer,
Satish Chandra,
Charles Sutton
Abstract:
We propose using natural language outlines as a novel modality and interaction surface for providing AI assistance to developers throughout the software development process. An NL outline for a code function comprises multiple statements written in concise prose, which partition the code and summarize its main ideas in the style of literate programming. Crucially, we find that modern LLMs can gene…
▽ More
We propose using natural language outlines as a novel modality and interaction surface for providing AI assistance to developers throughout the software development process. An NL outline for a code function comprises multiple statements written in concise prose, which partition the code and summarize its main ideas in the style of literate programming. Crucially, we find that modern LLMs can generate accurate and high-quality NL outlines in practice. Moreover, NL outlines enable a bidirectional sync between code and NL, allowing changes in one to be automatically reflected in the other. We discuss many use cases for NL outlines: they can accelerate understanding and navigation of code and diffs, simplify code maintenance, augment code search, steer code generation, and more. We then propose and compare multiple LLM prompting techniques for generating outlines and ask professional developers to judge outline quality. Finally, we present two case studies applying NL outlines toward code review and the difficult task of malware detection.
△ Less
Submitted 8 August, 2024;
originally announced August 2024.
-
Probabilistic Tracking with Deep Factors
Authors:
Fan Jiang,
Andrew Marmon,
Ildebrando De Courten,
Marc Rasi,
Frank Dellaert
Abstract:
In many applications of computer vision it is important to accurately estimate the trajectory of an object over time by fusing data from a number of sources, of which 2D and 3D imagery is only one. In this paper, we show how to use a deep feature encoding in conjunction with generative densities over the features in a factor-graph based, probabilistic tracking framework. We present a likelihood mo…
▽ More
In many applications of computer vision it is important to accurately estimate the trajectory of an object over time by fusing data from a number of sources, of which 2D and 3D imagery is only one. In this paper, we show how to use a deep feature encoding in conjunction with generative densities over the features in a factor-graph based, probabilistic tracking framework. We present a likelihood model that combines a learned feature encoder with generative densities over them, both trained in a supervised manner. We also experiment with directly inferring probability through the use of image classification models that feed into the likelihood formulation. These models are used to implement deep factors that are added to the factor graph to complement other factors that represent domain-specific knowledge such as motion models and/or other prior information. Factors are then optimized together in a non-linear least-squares tracking framework that takes the form of an Extended Kalman Smoother with a Gaussian prior. A key feature of our likelihood model is that it leverages the Lie group properties of the tracked target's pose to apply the feature encoding on an image patch, extracted through a differentiable warp function inspired by spatial transformer networks. To illustrate the proposed approach we evaluate it on a challenging social insect behavior dataset, and show that using deep features does outperform these earlier linear appearance models used in this setting.
△ Less
Submitted 2 December, 2021;
originally announced December 2021.
-
Swift for TensorFlow: A portable, flexible platform for deep learning
Authors:
Brennan Saeta,
Denys Shabalin,
Marc Rasi,
Brad Larson,
Xihui Wu,
Parker Schuh,
Michelle Casbon,
Daniel Zheng,
Saleem Abdulrasool,
Aleksandr Efremov,
Dave Abrahams,
Chris Lattner,
Richard Wei
Abstract:
Swift for TensorFlow is a deep learning platform that scales from mobile devices to clusters of hardware accelerators in data centers. It combines a language-integrated automatic differentiation system and multiple Tensor implementations within a modern ahead-of-time compiled language oriented around mutable value semantics. The resulting platform has been validated through use in over 30 deep lea…
▽ More
Swift for TensorFlow is a deep learning platform that scales from mobile devices to clusters of hardware accelerators in data centers. It combines a language-integrated automatic differentiation system and multiple Tensor implementations within a modern ahead-of-time compiled language oriented around mutable value semantics. The resulting platform has been validated through use in over 30 deep learning models and has been employed across data center and mobile applications.
△ Less
Submitted 25 February, 2021;
originally announced February 2021.
-
Trust Evaluation using an Improved Context Similarity Measurement
Authors:
Mohsen Raeesi,
Mohammad Amin Morid,
Mehdi Shajari
Abstract:
In context-aware trust evaluation, using ontology tree is a popular approach to represent the relation between contexts. Usually, similarity between two contexts is computed using these trees. Therefore, the performance of trust evaluation highly depends on the quality of ontology trees. Fairness or granularity consistency is one of the major limitations affecting the quality of ontology tree. Thi…
▽ More
In context-aware trust evaluation, using ontology tree is a popular approach to represent the relation between contexts. Usually, similarity between two contexts is computed using these trees. Therefore, the performance of trust evaluation highly depends on the quality of ontology trees. Fairness or granularity consistency is one of the major limitations affecting the quality of ontology tree. This limitation refers to inequality of semantic similarity in the most ontology trees. In other words, semantic similarity of every two adjacent nodes is unequal in these trees. It deteriorates the performance of contexts similarity computation. We overcome this limitation by weighting tree edges based on their semantic similarity. Weight of each edge is computed using Normalized Similarity Score (NSS) method. This method is based on frequencies of concepts (words) co-occurrences in the pages indexed by search engines. Our experiments represent the better performance of the proposed approach in comparison with established trust evaluation approaches. The suggested approach can enhance efficiency of any solution which models semantic relations by ontology tree.
△ Less
Submitted 17 April, 2014;
originally announced April 2014.
