Showing 1–8 of 8 results for author: Ward, W

Fine-Tuning Language Models Using Formal Methods Feedback

Authors: Yunhao Yang, Neel P. Bhatt, Tyler Ingebrand, William Ward, Steven Carr, Zhangyang Wang, Ufuk Topcu

Abstract: Although pre-trained language models encode generic knowledge beneficial for planning and control, they may fail to generate appropriate control policies for domain-specific tasks. Existing fine-tuning methods use human feedback to address this limitation, however, sourcing human feedback is labor intensive and costly. We present a fully automated approach to fine-tune pre-trained language models… ▽ More Although pre-trained language models encode generic knowledge beneficial for planning and control, they may fail to generate appropriate control policies for domain-specific tasks. Existing fine-tuning methods use human feedback to address this limitation, however, sourcing human feedback is labor intensive and costly. We present a fully automated approach to fine-tune pre-trained language models for applications in autonomous systems, bridging the gap between generic knowledge and domain-specific requirements while reducing cost. The method synthesizes automaton-based controllers from pre-trained models guided by natural language task descriptions. These controllers are verifiable against independently provided specifications within a world model, which can be abstract or obtained from a high-fidelity simulator. Controllers with high compliance with the desired specifications receive higher ranks, guiding the iterative fine-tuning process. We provide quantitative evidences, primarily in autonomous driving, to demonstrate the method's effectiveness across multiple tasks. The results indicate an improvement in percentage of specifications satisfied by the controller from 60% to 90%. △ Less

Submitted 27 October, 2023; originally announced October 2023.

My Science Tutor (MyST) -- A Large Corpus of Children's Conversational Speech

Authors: Sameer S. Pradhan, Ronald A. Cole, Wayne H. Ward

Abstract: This article describes the MyST corpus developed as part of the My Science Tutor project -- one of the largest collections of children's conversational speech comprising approximately 400 hours, spanning some 230K utterances across about 10.5K virtual tutor sessions by around 1.3K third, fourth and fifth grade students. 100K of all utterances have been transcribed thus far. The corpus is freely av… ▽ More This article describes the MyST corpus developed as part of the My Science Tutor project -- one of the largest collections of children's conversational speech comprising approximately 400 hours, spanning some 230K utterances across about 10.5K virtual tutor sessions by around 1.3K third, fourth and fifth grade students. 100K of all utterances have been transcribed thus far. The corpus is freely available (https://myst.cemantix.org) for non-commercial use using a creative commons license. It is also available for commercial use (https://boulderlearning.com/resources/myst-corpus/). To date, ten organizations have licensed the corpus for commercial use, and approximately 40 university and other not-for-profit research groups have downloaded the corpus. It is our hope that the corpus can be used to improve automatic speech recognition algorithms, build and evaluate conversational AI agents for education, and together help accelerate development of multimodal applications to improve children's excitement and learning about science, and help them learn remotely. △ Less

Submitted 23 September, 2023; originally announced September 2023.

Using Transformers to Provide Teachers with Personalized Feedback on their Classroom Discourse: The TalkMoves Application

Authors: Abhijit Suresh, Jennifer Jacobs, Vivian Lai, Chenhao Tan, Wayne Ward, James H. Martin, Tamara Sumner

Abstract: TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application combines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types o… ▽ More TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application combines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types of discourse aimed at broadening and deepening classroom conversations about mathematics. These specific discourse strategies are referred to as "talk moves" within the mathematics education community and prior research has documented the ways in which systematic use of these discourse strategies can positively impact student engagement and learning. In this article, we describe the TalkMoves application's cloud-based infrastructure for managing and processing classroom recordings, and its interface for providing teachers with feedback on their use of talk moves during individual teaching episodes. We present the series of model architectures we developed, and the studies we conducted, to develop our best-performing, transformer-based model (F1 = 79.3%). We also discuss several technical challenges that need to be addressed when working with real-world speech and language data from noisy K-12 classrooms. △ Less

Submitted 29 April, 2021; originally announced May 2021.

Comments: Presented at the AAAI 2021 Spring Symposium on Artificial Intelligence for K-12 Education

Cyclic Sequence Generators as Program Counters for High-Speed FPGA-based Processors

Authors: P. A. Suggate, R. W. Ward, T. C. A. Molteno

Abstract: This paper compares the performance of conventional radix-2 program counters with program counters based on Feedback Shift Registers (FSRs), a class of cyclic sequence generator. FSR counters have constant time scaling with bit-width, $N$, whereas FPGA-based radix-2 counters typically have $O(N)$ time-complexity due to the carry-chain. Program counter performance is measured by synthesis of standa… ▽ More This paper compares the performance of conventional radix-2 program counters with program counters based on Feedback Shift Registers (FSRs), a class of cyclic sequence generator. FSR counters have constant time scaling with bit-width, $N$, whereas FPGA-based radix-2 counters typically have $O(N)$ time-complexity due to the carry-chain. Program counter performance is measured by synthesis of standalone counter circuits, as well as synthesis of three FPGA-based processor designs modified to incorporate FSR program counters. Hybrid counters, combining both an FSR and a radix-2 counter, are presented as a solution to the potential cache-coherency issues of FSR program counters. Results show that high-speed processor designs benefit more from FSR program counters, allowing both greater operating frequency and the use of fewer logic resources. △ Less

Submitted 26 August, 2019; originally announced August 2019.

MSC Class: 68 ACM Class: C.1.3

Black-Box Inference for Non-Linear Latent Force Models

Authors: Wil O. C. Ward, Tom Ryder, Dennis Prangle, Mauricio A. Álvarez

Abstract: Latent force models are systems whereby there is a mechanistic model describing the dynamics of the system state, with some unknown forcing term that is approximated with a Gaussian process. If such dynamics are non-linear, it can be difficult to estimate the posterior state and forcing term jointly, particularly when there are system parameters that also need estimating. This paper uses black-box… ▽ More Latent force models are systems whereby there is a mechanistic model describing the dynamics of the system state, with some unknown forcing term that is approximated with a Gaussian process. If such dynamics are non-linear, it can be difficult to estimate the posterior state and forcing term jointly, particularly when there are system parameters that also need estimating. This paper uses black-box variational inference to jointly estimate the posterior, designing a multivariate extension to local inverse autoregressive flows as a flexible approximater of the system. We compare estimates on systems where the posterior is known, demonstrating the effectiveness of the approximation, and apply to problems with non-linear dynamics, multi-output systems and models with non-Gaussian likelihoods. △ Less

Submitted 4 November, 2019; v1 submitted 21 June, 2019; originally announced June 2019.

Comments: 13 pages plus references and supplementary

Variational bridge constructs for approximate Gaussian process regression

Authors: Wil O C Ward, Mauricio A Álvarez

Abstract: This paper introduces a method to approximate Gaussian process regression by representing the problem as a stochastic differential equation and using variational inference to approximate solutions. The approximations are compared with full GP regression and generated paths are demonstrated to be indistinguishable from GP samples. We show that the approach extends easily to non-linear dynamics and… ▽ More This paper introduces a method to approximate Gaussian process regression by representing the problem as a stochastic differential equation and using variational inference to approximate solutions. The approximations are compared with full GP regression and generated paths are demonstrated to be indistinguishable from GP samples. We show that the approach extends easily to non-linear dynamics and discuss extensions to which the approach can be easily applied. △ Less

Submitted 7 January, 2019; originally announced January 2019.

Comments: 4 pages, presented at BNP@NeurIPS 2018

Non-linear process convolutions for multi-output Gaussian processes

Authors: Mauricio A. Álvarez, Wil O. C. Ward, Cristian Guarnizo

Abstract: The paper introduces a non-linear version of the process convolution formalism for building covariance functions for multi-output Gaussian processes. The non-linearity is introduced via Volterra series, one series per each output. We provide closed-form expressions for the mean function and the covariance function of the approximated Gaussian process at the output of the Volterra series. The mean… ▽ More The paper introduces a non-linear version of the process convolution formalism for building covariance functions for multi-output Gaussian processes. The non-linearity is introduced via Volterra series, one series per each output. We provide closed-form expressions for the mean function and the covariance function of the approximated Gaussian process at the output of the Volterra series. The mean function and covariance function for the joint Gaussian process are derived using formulae for the product moments of Gaussian variables. We compare the performance of the non-linear model against the classical process convolution approach in one synthetic dataset and two real datasets. △ Less

Submitted 27 February, 2019; v1 submitted 10 October, 2018; originally announced October 2018.

Comments: 16 pages plus 2 page supplementary. Accepted to AISTATS 2019

Tensor Based Second Order Variational Model for Image Reconstruction

Authors: Jinming Duan, Wil OC Ward, Luke Sibbett, Zhenkuan Pan, Li Bai

Abstract: Second order total variation (SOTV) models have advantages for image reconstruction over their first order counterparts including their ability to remove the staircase artefact in the reconstructed image, but they tend to blur the reconstructed image. To overcome this drawback, we introduce a new Tensor Weighted Second Order (TWSO) model for image reconstruction. Specifically, we develop a novel r… ▽ More Second order total variation (SOTV) models have advantages for image reconstruction over their first order counterparts including their ability to remove the staircase artefact in the reconstructed image, but they tend to blur the reconstructed image. To overcome this drawback, we introduce a new Tensor Weighted Second Order (TWSO) model for image reconstruction. Specifically, we develop a novel regulariser for the SOTV model that uses the Frobenius norm of the product of the SOTV Hessian matrix and the anisotropic tensor. We then adapt the alternating direction method of multipliers (ADMM) to solve the proposed model by breaking down the original problem into several subproblems. All the subproblems have closed-forms and can thus be solved efficiently. The proposed method is compared with a range of state-of-the-art approaches such as tensor-based anisotropic diffusion, total generalised variation, Euler's elastica, etc. Numerical experimental results of the method on both synthetic and real images from the Berkeley database BSDS500 demonstrate that the proposed method eliminates both the staircase and blurring effects and outperforms the existing approaches for image inpainting and denoising applications. △ Less

Submitted 27 September, 2016; originally announced September 2016.