-
Fine-Tuning Language Models Using Formal Methods Feedback
Authors:
Yunhao Yang,
Neel P. Bhatt,
Tyler Ingebrand,
William Ward,
Steven Carr,
Zhangyang Wang,
Ufuk Topcu
Abstract:
Although pre-trained language models encode generic knowledge beneficial for planning and control, they may fail to generate appropriate control policies for domain-specific tasks. Existing fine-tuning methods use human feedback to address this limitation, however, sourcing human feedback is labor intensive and costly. We present a fully automated approach to fine-tune pre-trained language models…
▽ More
Although pre-trained language models encode generic knowledge beneficial for planning and control, they may fail to generate appropriate control policies for domain-specific tasks. Existing fine-tuning methods use human feedback to address this limitation, however, sourcing human feedback is labor intensive and costly. We present a fully automated approach to fine-tune pre-trained language models for applications in autonomous systems, bridging the gap between generic knowledge and domain-specific requirements while reducing cost. The method synthesizes automaton-based controllers from pre-trained models guided by natural language task descriptions. These controllers are verifiable against independently provided specifications within a world model, which can be abstract or obtained from a high-fidelity simulator. Controllers with high compliance with the desired specifications receive higher ranks, guiding the iterative fine-tuning process. We provide quantitative evidences, primarily in autonomous driving, to demonstrate the method's effectiveness across multiple tasks. The results indicate an improvement in percentage of specifications satisfied by the controller from 60% to 90%.
△ Less
Submitted 27 October, 2023;
originally announced October 2023.
-
My Science Tutor (MyST) -- A Large Corpus of Children's Conversational Speech
Authors:
Sameer S. Pradhan,
Ronald A. Cole,
Wayne H. Ward
Abstract:
This article describes the MyST corpus developed as part of the My Science Tutor project -- one of the largest collections of children's conversational speech comprising approximately 400 hours, spanning some 230K utterances across about 10.5K virtual tutor sessions by around 1.3K third, fourth and fifth grade students. 100K of all utterances have been transcribed thus far. The corpus is freely av…
▽ More
This article describes the MyST corpus developed as part of the My Science Tutor project -- one of the largest collections of children's conversational speech comprising approximately 400 hours, spanning some 230K utterances across about 10.5K virtual tutor sessions by around 1.3K third, fourth and fifth grade students. 100K of all utterances have been transcribed thus far. The corpus is freely available (https://myst.cemantix.org) for non-commercial use using a creative commons license. It is also available for commercial use (https://boulderlearning.com/resources/myst-corpus/). To date, ten organizations have licensed the corpus for commercial use, and approximately 40 university and other not-for-profit research groups have downloaded the corpus. It is our hope that the corpus can be used to improve automatic speech recognition algorithms, build and evaluate conversational AI agents for education, and together help accelerate development of multimodal applications to improve children's excitement and learning about science, and help them learn remotely.
△ Less
Submitted 23 September, 2023;
originally announced September 2023.
-
Using Transformers to Provide Teachers with Personalized Feedback on their Classroom Discourse: The TalkMoves Application
Authors:
Abhijit Suresh,
Jennifer Jacobs,
Vivian Lai,
Chenhao Tan,
Wayne Ward,
James H. Martin,
Tamara Sumner
Abstract:
TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application combines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types o…
▽ More
TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application combines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types of discourse aimed at broadening and deepening classroom conversations about mathematics. These specific discourse strategies are referred to as "talk moves" within the mathematics education community and prior research has documented the ways in which systematic use of these discourse strategies can positively impact student engagement and learning. In this article, we describe the TalkMoves application's cloud-based infrastructure for managing and processing classroom recordings, and its interface for providing teachers with feedback on their use of talk moves during individual teaching episodes. We present the series of model architectures we developed, and the studies we conducted, to develop our best-performing, transformer-based model (F1 = 79.3%). We also discuss several technical challenges that need to be addressed when working with real-world speech and language data from noisy K-12 classrooms.
△ Less
Submitted 29 April, 2021;
originally announced May 2021.
-
Cyclic Sequence Generators as Program Counters for High-Speed FPGA-based Processors
Authors:
P. A. Suggate,
R. W. Ward,
T. C. A. Molteno
Abstract:
This paper compares the performance of conventional radix-2 program counters with program counters based on Feedback Shift Registers (FSRs), a class of cyclic sequence generator. FSR counters have constant time scaling with bit-width, $N$, whereas FPGA-based radix-2 counters typically have $O(N)$ time-complexity due to the carry-chain. Program counter performance is measured by synthesis of standa…
▽ More
This paper compares the performance of conventional radix-2 program counters with program counters based on Feedback Shift Registers (FSRs), a class of cyclic sequence generator. FSR counters have constant time scaling with bit-width, $N$, whereas FPGA-based radix-2 counters typically have $O(N)$ time-complexity due to the carry-chain. Program counter performance is measured by synthesis of standalone counter circuits, as well as synthesis of three FPGA-based processor designs modified to incorporate FSR program counters. Hybrid counters, combining both an FSR and a radix-2 counter, are presented as a solution to the potential cache-coherency issues of FSR program counters. Results show that high-speed processor designs benefit more from FSR program counters, allowing both greater operating frequency and the use of fewer logic resources.
△ Less
Submitted 26 August, 2019;
originally announced August 2019.
-
Black-Box Inference for Non-Linear Latent Force Models
Authors:
Wil O. C. Ward,
Tom Ryder,
Dennis Prangle,
Mauricio A. Álvarez
Abstract:
Latent force models are systems whereby there is a mechanistic model describing the dynamics of the system state, with some unknown forcing term that is approximated with a Gaussian process. If such dynamics are non-linear, it can be difficult to estimate the posterior state and forcing term jointly, particularly when there are system parameters that also need estimating. This paper uses black-box…
▽ More
Latent force models are systems whereby there is a mechanistic model describing the dynamics of the system state, with some unknown forcing term that is approximated with a Gaussian process. If such dynamics are non-linear, it can be difficult to estimate the posterior state and forcing term jointly, particularly when there are system parameters that also need estimating. This paper uses black-box variational inference to jointly estimate the posterior, designing a multivariate extension to local inverse autoregressive flows as a flexible approximater of the system. We compare estimates on systems where the posterior is known, demonstrating the effectiveness of the approximation, and apply to problems with non-linear dynamics, multi-output systems and models with non-Gaussian likelihoods.
△ Less
Submitted 4 November, 2019; v1 submitted 21 June, 2019;
originally announced June 2019.
-
Variational bridge constructs for approximate Gaussian process regression
Authors:
Wil O C Ward,
Mauricio A Álvarez
Abstract:
This paper introduces a method to approximate Gaussian process regression by representing the problem as a stochastic differential equation and using variational inference to approximate solutions. The approximations are compared with full GP regression and generated paths are demonstrated to be indistinguishable from GP samples. We show that the approach extends easily to non-linear dynamics and…
▽ More
This paper introduces a method to approximate Gaussian process regression by representing the problem as a stochastic differential equation and using variational inference to approximate solutions. The approximations are compared with full GP regression and generated paths are demonstrated to be indistinguishable from GP samples. We show that the approach extends easily to non-linear dynamics and discuss extensions to which the approach can be easily applied.
△ Less
Submitted 7 January, 2019;
originally announced January 2019.
-
Non-linear process convolutions for multi-output Gaussian processes
Authors:
Mauricio A. Álvarez,
Wil O. C. Ward,
Cristian Guarnizo
Abstract:
The paper introduces a non-linear version of the process convolution formalism for building covariance functions for multi-output Gaussian processes. The non-linearity is introduced via Volterra series, one series per each output. We provide closed-form expressions for the mean function and the covariance function of the approximated Gaussian process at the output of the Volterra series. The mean…
▽ More
The paper introduces a non-linear version of the process convolution formalism for building covariance functions for multi-output Gaussian processes. The non-linearity is introduced via Volterra series, one series per each output. We provide closed-form expressions for the mean function and the covariance function of the approximated Gaussian process at the output of the Volterra series. The mean function and covariance function for the joint Gaussian process are derived using formulae for the product moments of Gaussian variables. We compare the performance of the non-linear model against the classical process convolution approach in one synthetic dataset and two real datasets.
△ Less
Submitted 27 February, 2019; v1 submitted 10 October, 2018;
originally announced October 2018.
-
Tensor Based Second Order Variational Model for Image Reconstruction
Authors:
Jinming Duan,
Wil OC Ward,
Luke Sibbett,
Zhenkuan Pan,
Li Bai
Abstract:
Second order total variation (SOTV) models have advantages for image reconstruction over their first order counterparts including their ability to remove the staircase artefact in the reconstructed image, but they tend to blur the reconstructed image. To overcome this drawback, we introduce a new Tensor Weighted Second Order (TWSO) model for image reconstruction. Specifically, we develop a novel r…
▽ More
Second order total variation (SOTV) models have advantages for image reconstruction over their first order counterparts including their ability to remove the staircase artefact in the reconstructed image, but they tend to blur the reconstructed image. To overcome this drawback, we introduce a new Tensor Weighted Second Order (TWSO) model for image reconstruction. Specifically, we develop a novel regulariser for the SOTV model that uses the Frobenius norm of the product of the SOTV Hessian matrix and the anisotropic tensor. We then adapt the alternating direction method of multipliers (ADMM) to solve the proposed model by breaking down the original problem into several subproblems. All the subproblems have closed-forms and can thus be solved efficiently. The proposed method is compared with a range of state-of-the-art approaches such as tensor-based anisotropic diffusion, total generalised variation, Euler's elastica, etc. Numerical experimental results of the method on both synthetic and real images from the Berkeley database BSDS500 demonstrate that the proposed method eliminates both the staircase and blurring effects and outperforms the existing approaches for image inpainting and denoising applications.
△ Less
Submitted 27 September, 2016;
originally announced September 2016.