-
Disfluency Detection with Unlabeled Data and Small BERT Models
Authors:
Johann C. Rocholl,
Vicky Zayats,
Daniel D. Walker,
Noah B. Murad,
Aaron Schneider,
Daniel J. Liebling
Abstract:
Disfluency detection models now approach high accuracy on English text. However, little exploration has been done in improving the size and inference time of the model. At the same time, automatic speech recognition (ASR) models are moving from server-side inference to local, on-device inference. Supporting models in the transcription pipeline (like disfluency detection) must follow suit. In this…
▽ More
Disfluency detection models now approach high accuracy on English text. However, little exploration has been done in improving the size and inference time of the model. At the same time, automatic speech recognition (ASR) models are moving from server-side inference to local, on-device inference. Supporting models in the transcription pipeline (like disfluency detection) must follow suit. In this work we concentrate on the disfluency detection task, focusing on small, fast, on-device models based on the BERT architecture. We demonstrate it is possible to train disfluency detection models as small as 1.3 MiB, while retaining high performance. We build on previous work that showed the benefit of data augmentation approaches such as self-training. Then, we evaluate the effect of domain mismatch between conversational and written text on model performance. We find that domain adaptation and data augmentation strategies have a more pronounced effect on these smaller models, as compared to conventional BERT models.
△ Less
Submitted 27 July, 2021; v1 submitted 21 April, 2021;
originally announced April 2021.
-
AMPL: A Data-Driven Modeling Pipeline for Drug Discovery
Authors:
Amanda J. Minnich,
Kevin McLoughlin,
Margaret Tse,
Jason Deng,
Andrew Weber,
Neha Murad,
Benjamin D. Madej,
Bharath Ramsundar,
Tom Rush,
Stacie Calad-Thomson,
Jim Brase,
Jonathan E. Allen
Abstract:
One of the key requirements for incorporating machine learning into the drug discovery process is complete reproducibility and traceability of the model building and evaluation process. With this in mind, we have developed an end-to-end modular and extensible software pipeline for building and sharing machine learning models that predict key pharma-relevant parameters. The ATOM Modeling PipeLine,…
▽ More
One of the key requirements for incorporating machine learning into the drug discovery process is complete reproducibility and traceability of the model building and evaluation process. With this in mind, we have developed an end-to-end modular and extensible software pipeline for building and sharing machine learning models that predict key pharma-relevant parameters. The ATOM Modeling PipeLine, or AMPL, extends the functionality of the open source library DeepChem and supports an array of machine learning and molecular featurization tools. We have benchmarked AMPL on a large collection of pharmaceutical datasets covering a wide range of parameters. As a result of these comprehensive experiments, we have found that physicochemical descriptors and deep learning-based graph representations significantly outperform traditional fingerprints in the characterization of molecular features. We have also found that dataset size is directly correlated to prediction performance, and that single-task deep learning models only outperform shallow learners if there is sufficient data. Likewise, dataset size has a direct impact on model predictivity, independent of comprehensive hyperparameter model tuning. Our findings point to the need for public dataset integration or multi-task/transfer learning approaches. Lastly, we found that uncertainty quantification (UQ) analysis may help identify model error; however, efficacy of UQ to filter predictions varies considerably between datasets and featurization/model types. AMPL is open source and available for download at http://github.com/ATOMconsortium/AMPL.
△ Less
Submitted 13 November, 2019; v1 submitted 12 November, 2019;
originally announced November 2019.
-
Integrating Over Sea Radio Channel for Sea Turtles Localization in the Indian Ocean
Authors:
Loic Guegan,
Nour Murad,
Jean Lebreton,
Sylvain Bonhommeau
Abstract:
This paper deals with the modeling of over sea radio channel with the aim of establishing sea turtle localization off the coast of Reunion Island but also on Europa Island in the Mozambique Channel. In order to model this radio channel, we are making a measurement protocol. In a first approach, measurements of turtle trajectory were done over land and finally it will be conducted over sea. We have…
▽ More
This paper deals with the modeling of over sea radio channel with the aim of establishing sea turtle localization off the coast of Reunion Island but also on Europa Island in the Mozambique Channel. In order to model this radio channel, we are making a measurement protocol. In a first approach, measurements of turtle trajectory were done over land and finally it will be conducted over sea. We have scheduled an over sea measurement campaign in the middle of June. This paper shows a signal cross correlation technique used to characterize the over sea propagation channel.
△ Less
Submitted 10 November, 2017;
originally announced December 2017.
-
Where is My Next Hop ? The Case of Indian Ocean Islands
Authors:
Xavier Nicolay,
Rehan Noordally,
Pascal Anelli,
Nour Mohammad Murad,
Tahiry Razafindralambo
Abstract:
Internet has become a foundation of our modern society. However, all regions or countries do not have the same Internet access regarding quality especially in the Indian Ocean Area (IOA). To improve this quality it is important to have a deep knowledge of the Internet physical and logical topology and associated performance. However, these knowledges are not shared by Internet service providers. I…
▽ More
Internet has become a foundation of our modern society. However, all regions or countries do not have the same Internet access regarding quality especially in the Indian Ocean Area (IOA). To improve this quality it is important to have a deep knowledge of the Internet physical and logical topology and associated performance. However, these knowledges are not shared by Internet service providers. In this paper, we describe a large scale measurement study in which we deploy probes in different IOA countries, we generate network traces, develop a tool to extract useful information and analyze these information. We show that most of the IOA traffic exits through one point even if there exists multiple exit points.
△ Less
Submitted 28 September, 2017; v1 submitted 21 July, 2017;
originally announced July 2017.
-
Implementation of a Wake-up Radio Cross-Layer Protocol in OMNeT++ / MiXiM
Authors:
Jean Lebreton,
Nour Murad
Abstract:
This paper presents the DoRa protocol, which is a new cross-layer protocol for handling the double radio of nodes in wake-up radio scenario. The implementation details in OMNET++/MiXiM are also given, with a focus on the implemented MAC layers. The main goal of the DoRa protocol is to reduce energy consumption in wireless sensor network, by taking full advantage of the passive wake-up scheme. The…
▽ More
This paper presents the DoRa protocol, which is a new cross-layer protocol for handling the double radio of nodes in wake-up radio scenario. The implementation details in OMNET++/MiXiM are also given, with a focus on the implemented MAC layers. The main goal of the DoRa protocol is to reduce energy consumption in wireless sensor network, by taking full advantage of the passive wake-up scheme. The performance of the DoRa protocol is then evaluated and results are compared with B-MAC and IEEE 802.15.4 protocols.
△ Less
Submitted 11 September, 2015;
originally announced September 2015.
