Showing 1–20 of 20 results for author: Santos, J F

Managing O-RAN Networks: xApp Development from Zero to Hero

Authors: Joao F. Santos, Alexandre Huff, Daniel Campos, Kleber V. Cardoso, Cristiano B. Both, Luiz A. DaSilva

Abstract: The Open Radio Access Network (O-RAN) Alliance proposes an open architecture that disaggregates the RAN and supports executing custom control logic in near-real time from third-party applications, the xApps. Despite O-RAN's efforts, the creation of xApps remains a complex and time-consuming endeavor, aggravated by the sometimes fragmented, outdated, or deprecated documentation from the O-RAN Softw… ▽ More The Open Radio Access Network (O-RAN) Alliance proposes an open architecture that disaggregates the RAN and supports executing custom control logic in near-real time from third-party applications, the xApps. Despite O-RAN's efforts, the creation of xApps remains a complex and time-consuming endeavor, aggravated by the sometimes fragmented, outdated, or deprecated documentation from the O-RAN Software Community (OSC). These challenges hinder academia and industry from developing and validating solutions and algorithms on O-RAN networks. This tutorial addresses this gap by providing the first comprehensive guide for developing xApps to manage the O-RAN ecosystem from theory to practice. We provide a thorough theoretical foundation of the O-RAN architecture and detail the functionality offered by Near Real-Time RAN Intelligent Controller (Near-RT RIC) components. We examine the xApp design and configuration. We explore the xApp lifecycle and demonstrate how to deploy and manage xApps on a Near-RT RIC. We address the xApps' interfaces and capabilities, accompanied by practical examples. We provide comprehensive details on how xApps can control the RAN. We discuss debugging strategies and good practices to aid the xApp developers in testing their xApps. Finally, we review the current landscape and open challenges for creating xApps. △ Less

Submitted 4 August, 2024; v1 submitted 12 July, 2024; originally announced July 2024.

Decentralized Multi-Party Multi-Network AI for Global Deployment of 6G Wireless Systems

Authors: Merim Dzaferagic, Marco Ruffini, Nina Slamnik-Krijestorac, Joao F. Santos, Johann Marquez-Barja, Christos Tranoris, Spyros Denazis, Thomas Kyriakakis, Panagiotis Karafotis, Luiz DaSilva, Shashi Raj Pandey, Junya Shiraishi, Petar Popovski, Soren Kejser Jensen, Christian Thomsen, Torben Bach Pedersen, Holger Claussen, Jinfeng Du, Gil Zussman, Tingjun Chen, Yiran Chen, Seshu Tirupathi, Ivan Seskar, Daniel Kilper

Abstract: Multiple visions of 6G networks elicit Artificial Intelligence (AI) as a central, native element. When 6G systems are deployed at a large scale, end-to-end AI-based solutions will necessarily have to encompass both the radio and the fiber-optical domain. This paper introduces the Decentralized Multi-Party, Multi-Network AI (DMMAI) framework for integrating AI into 6G networks deployed at scale. DM… ▽ More Multiple visions of 6G networks elicit Artificial Intelligence (AI) as a central, native element. When 6G systems are deployed at a large scale, end-to-end AI-based solutions will necessarily have to encompass both the radio and the fiber-optical domain. This paper introduces the Decentralized Multi-Party, Multi-Network AI (DMMAI) framework for integrating AI into 6G networks deployed at scale. DMMAI harmonizes AI-driven controls across diverse network platforms and thus facilitates networks that autonomously configure, monitor, and repair themselves. This is particularly crucial at the network edge, where advanced applications meet heightened functionality and security demands. The radio/optical integration is vital due to the current compartmentalization of AI research within these domains, which lacks a comprehensive understanding of their interaction. Our approach explores multi-network orchestration and AI control integration, filling a critical gap in standardized frameworks for AI-driven coordination in 6G networks. The DMMAI framework is a step towards a global standard for AI in 6G, aiming to establish reference use cases, data and model management methods, and benchmarking platforms for future AI/ML solutions. △ Less

Submitted 15 April, 2024; originally announced July 2024.

Blockchain-based Rental Documentation Management with Audit Support

Authors: João F. Santos, Miguel Correia, Tiago R. Dias

Abstract: Document management in the rental market is a critical process to ensure the accuracy of financial transactions and regulatory compliance in the sector. In Portugal, the challenges include the complexity of legislation, particularly GDPR non-compliance, lack of transparency, and bureaucratic process inefficiency. With this in mind, a solution based on Hyperledger Fabric, a blockchain platform, is… ▽ More Document management in the rental market is a critical process to ensure the accuracy of financial transactions and regulatory compliance in the sector. In Portugal, the challenges include the complexity of legislation, particularly GDPR non-compliance, lack of transparency, and bureaucratic process inefficiency. With this in mind, a solution based on Hyperledger Fabric, a blockchain platform, is presented for the implementation of a document management system for the rental process. This system oversees the rental process, which consists of three phases: the application for a property by the prospective tenant through the upload of necessary documents, acceptance/rejection by the landlord of various received applications, and the creation of a report by the system, which only the auditor can request and view. The system smart contract records metadata associated with the documents (hash, owner) and coordinates requests for file access by landlords to prospective tenants. Thus, the system is responsible for creating immutable and traceable records of the entire process. The underlying platform serves as the foundation for conducting future audits. After the landlord verifies the files and accepts the rental proposal, any authorised auditor can request a report for a property by accessing the records through the final report, which includes all events that occurred during the process. △ Less

Submitted 9 February, 2024; originally announced February 2024.

Comments: 10 pages

VANI: Very-lightweight Accent-controllable TTS for Native and Non-native speakers with Identity Preservation

Authors: Rohan Badlani, Akshit Arora, Subhankar Ghosh, Rafael Valle, Kevin J. Shih, João Felipe Santos, Boris Ginsburg, Bryan Catanzaro

Abstract: We introduce VANI, a very lightweight multi-lingual accent controllable speech synthesis system. Our model builds upon disentanglement strategies proposed in RADMMM and supports explicit control of accent, language, speaker and fine-grained $F_0$ and energy features for speech synthesis. We utilize the Indic languages dataset, released for LIMMITS 2023 as part of ICASSP Signal Processing Grand Cha… ▽ More We introduce VANI, a very lightweight multi-lingual accent controllable speech synthesis system. Our model builds upon disentanglement strategies proposed in RADMMM and supports explicit control of accent, language, speaker and fine-grained $F_0$ and energy features for speech synthesis. We utilize the Indic languages dataset, released for LIMMITS 2023 as part of ICASSP Signal Processing Grand Challenge, to synthesize speech in 3 different languages. Our model supports transferring the language of a speaker while retaining their voice and the native accent of the target language. We utilize the large-parameter RADMMM model for Track $1$ and lightweight VANI model for Track $2$ and $3$ of the competition. △ Less

Submitted 13 March, 2023; originally announced March 2023.

Comments: Presentation accepted at ICASSP 2023

Multilingual Multiaccented Multispeaker TTS with RADTTS

Authors: Rohan Badlani, Rafael Valle, Kevin J. Shih, João Felipe Santos, Siddharth Gururani, Bryan Catanzaro

Abstract: We work to create a multilingual speech synthesis system which can generate speech with the proper accent while retaining the characteristics of an individual voice. This is challenging to do because it is expensive to obtain bilingual training data in multiple languages, and the lack of such data results in strong correlations that entangle speaker, language, and accent, resulting in poor transfe… ▽ More We work to create a multilingual speech synthesis system which can generate speech with the proper accent while retaining the characteristics of an individual voice. This is challenging to do because it is expensive to obtain bilingual training data in multiple languages, and the lack of such data results in strong correlations that entangle speaker, language, and accent, resulting in poor transfer capabilities. To overcome this, we present a multilingual, multiaccented, multispeaker speech synthesis model based on RADTTS with explicit control over accent, language, speaker and fine-grained $F_0$ and energy features. Our proposed model does not rely on bilingual training data. We demonstrate an ability to control synthesized accent for any speaker in an open-source dataset comprising of 7 accents. Human subjective evaluation demonstrates that our model can better retain a speaker's voice and accent quality than controlled baselines while synthesizing fluent speech in all target languages and accents in our dataset. △ Less

Submitted 24 January, 2023; originally announced January 2023.

Comments: 5 pages, submitted to ICASSP 2023

Study of JavaScript Static Analysis Tools for Vulnerability Detection in Node.js Packages

Authors: Tiago Brito, Mafalda Ferreira, Miguel Monteiro, Pedro Lopes, Miguel Barros, José Fragoso Santos, Nuno Santos

Abstract: With the emergence of the Node.js ecosystem, JavaScript has become a widely-used programming language for implementing server-side web applications. In this paper, we present the first empirical study of static code analysis tools for detecting vulnerabilities in Node.js code. To conduct a comprehensive tool evaluation, we created the largest known curated dataset of Node.js code vulnerabilities.… ▽ More With the emergence of the Node.js ecosystem, JavaScript has become a widely-used programming language for implementing server-side web applications. In this paper, we present the first empirical study of static code analysis tools for detecting vulnerabilities in Node.js code. To conduct a comprehensive tool evaluation, we created the largest known curated dataset of Node.js code vulnerabilities. We characterized and annotated a set of 957 vulnerabilities by analyzing information contained in npm advisory reports. We tested nine different tools and found that many important vulnerabilities appearing in the OWASP Top-10 are not detected by any tool. The three best performing tools combined only detect up to 57.6% of all vulnerabilities in the dataset, but at a very low precision of 0.11%. Our curated dataset offers a new benchmark to help characterize existing Node.js code vulnerabilities and foster the development of better vulnerability detection tools for Node.js code. △ Less

Submitted 12 January, 2023; originally announced January 2023.

TestSelector: Automatic Test Suite Selection for Student Projects -- Extended Version

Authors: Filipe Marques, António Morgado, José Fragoso Santos, Mikoláš Janota

Abstract: Computer Science course instructors routinely have to create comprehensive test suites to assess programming assignments. The creation of such test suites is typically not trivial as it involves selecting a limited number of tests from a set of (semi-)randomly generated ones. Manual strategies for test selection do not scale when considering large testing inputs needed, for instance, for the asses… ▽ More Computer Science course instructors routinely have to create comprehensive test suites to assess programming assignments. The creation of such test suites is typically not trivial as it involves selecting a limited number of tests from a set of (semi-)randomly generated ones. Manual strategies for test selection do not scale when considering large testing inputs needed, for instance, for the assessment of algorithms exercises. To facilitate this process, we present TestSelector, a new framework for automatic selection of optimal test suites for student projects. The key advantage of TestSelector over existing approaches is that it is easily extensible with arbitrarily complex code coverage measures, not requiring these measures to be encoded into the logic of an exact constraint solver. We demonstrate the flexibility of TestSelector by extending it with support for a range of classical code coverage measures and using it to select test suites for a number of real-world algorithms projects, further showing that the selected test suites outperform randomly selected ones in finding bugs in students' code. △ Less

Submitted 19 July, 2022; originally announced July 2022.

Comments: 19 pages

Wasmati: An Efficient Static Vulnerability Scanner for WebAssembly

Authors: Tiago Brito, Pedro Lopes, Nuno Santos, José Fragoso Santos

Abstract: WebAssembly is a new binary instruction format that allows targeted compiled code written in high-level languages to be executed with near-native speed by the browser's JavaScript engine. However, given that WebAssembly binaries can be compiled from unsafe languages like C/C++, classical code vulnerabilities such as buffer overflows or format strings can be transferred over from the original progr… ▽ More WebAssembly is a new binary instruction format that allows targeted compiled code written in high-level languages to be executed with near-native speed by the browser's JavaScript engine. However, given that WebAssembly binaries can be compiled from unsafe languages like C/C++, classical code vulnerabilities such as buffer overflows or format strings can be transferred over from the original programs down to the cross-compiled binaries. As a result, this possibility of incorporating vulnerabilities in WebAssembly modules has widened the attack surface of modern web applications. This paper presents Wasmati, a static analysis tool for finding security vulnerabilities in WebAssembly binaries. It is based on the generation of a code property graph (CPG), a program representation previously adopted for detecting vulnerabilities in various languages but hitherto unapplied to WebAssembly. We formalize the definition of CPG for WebAssembly, introduce techniques to generate CPG for complex WebAssembly, and present four different query specification languages for finding vulnerabilities by traversing a program's CPG. We implemented ten queries capturing different vulnerability types and extensively tested Wasmati on four heterogeneous datasets. We show that Wasmati can scale the generation of CPGs for large real-world applications and can efficiently find vulnerabilities for all our query types. We have also tested our tool on WebAssembly binaries collected in the wild and identified several potential vulnerabilities, some of which we have manually confirmed to exist unless the enclosing application properly sanitizes the interaction with such affected binaries. △ Less

Submitted 26 April, 2022; originally announced April 2022.

Comments: Computers & Security

Generative Modeling for Low Dimensional Speech Attributes with Neural Spline Flows

Authors: Kevin J. Shih, Rafael Valle, Rohan Badlani, João Felipe Santos, Bryan Catanzaro

Abstract: Despite recent advances in generative modeling for text-to-speech synthesis, these models do not yet have the same fine-grained adjustability of pitch-conditioned deterministic models such as FastPitch and FastSpeech2. Pitch information is not only low-dimensional, but also discontinuous, making it particularly difficult to model in a generative setting. Our work explores several techniques for ha… ▽ More Despite recent advances in generative modeling for text-to-speech synthesis, these models do not yet have the same fine-grained adjustability of pitch-conditioned deterministic models such as FastPitch and FastSpeech2. Pitch information is not only low-dimensional, but also discontinuous, making it particularly difficult to model in a generative setting. Our work explores several techniques for handling the aforementioned issues in the context of Normalizing Flow models. We also find this problem to be very well suited for Neural Spline flows, which is a highly expressive alternative to the more common affine-coupling mechanism in Normalizing Flows. △ Less

Submitted 27 June, 2022; v1 submitted 3 March, 2022; originally announced March 2022.

Comments: 22 pages, 11 figures, 3 tables

Gillian: A Multi-Language Platform for Unified Symbolic Analysis

Authors: Petar Maksimović, José Fragoso Santos, Sacha-Élie Ayoun, Philippa Gardner

Abstract: This is an evolving document describing the meta-theory, the implementation, and the instantiations of Gillian, a multi-language symbolic analysis platform. This is an evolving document describing the meta-theory, the implementation, and the instantiations of Gillian, a multi-language symbolic analysis platform. △ Less

Submitted 12 July, 2021; v1 submitted 31 May, 2021; originally announced May 2021.

Radio Access Technology Characterisation Through Object Detection

Authors: Erika Fonseca, Joao F. Santos, Francisco Paisana, Luiz A. DaSilva

Abstract: \ac{RAT} classification and monitoring are essential for efficient coexistence of different communication systems in shared spectrum. Shared spectrum, including operation in license-exempt bands, is envisioned in the \ac{5G} standards (e.g., 3GPP Rel. 16). In this paper, we propose a \ac{ML} approach to characterise the spectrum utilisation and facilitate the dynamic access to it. Recent advances… ▽ More \ac{RAT} classification and monitoring are essential for efficient coexistence of different communication systems in shared spectrum. Shared spectrum, including operation in license-exempt bands, is envisioned in the \ac{5G} standards (e.g., 3GPP Rel. 16). In this paper, we propose a \ac{ML} approach to characterise the spectrum utilisation and facilitate the dynamic access to it. Recent advances in \acp{CNN} enable us to perform waveform classification by processing spectrograms as images. In contrast to other \ac{ML} methods that can only provide the class of the monitored \acp{RAT}, the solution we propose can recognise not only different \acp{RAT} in shared spectrum, but also identify critical parameters such as inter-frame duration, frame duration, centre frequency, and signal bandwidth by using object detection and a feature extraction module to extract features from spectrograms. We have implemented and evaluated our solution using a dataset of commercial transmissions, as well as in a \ac{SDR} testbed environment. The scenario evaluated was the coexistence of WiFi and LTE transmissions in shared spectrum. Our results show that our approach has an accuracy of 96\% in the classification of \acp{RAT} from a dataset that captures transmissions of regular user communications. It also shows that the extracted features can be precise within a margin of 2\%, %of the size of the image, and is capable of detect above 94\% of objects under a broad range of transmission power levels and interference conditions. △ Less

Submitted 27 July, 2020; originally announced July 2020.

Implementing a Quantum Coin Scheme

Authors: Hazel Murray, Jerry Horgan, Joao F. Santos, David Malone, Harun Siljak

Abstract: Quantum computing has the power to break current cryptographic systems, disrupting online banking, shopping, data storage and communications. Quantum computing also has the power to support stronger more resistant technologies. In this paper, we describe a digital cash scheme created by Dmitry Gavinsky, which utilises the capability of quantum computing. We contribute by setting out the methods fo… ▽ More Quantum computing has the power to break current cryptographic systems, disrupting online banking, shopping, data storage and communications. Quantum computing also has the power to support stronger more resistant technologies. In this paper, we describe a digital cash scheme created by Dmitry Gavinsky, which utilises the capability of quantum computing. We contribute by setting out the methods for implementing this scheme. For both the creation and verification of quantum coins we convert the algebraic steps into computing steps. As part of this, we describe the methods used to convert information stored on classical bits to information stored on quantum bits. △ Less

Submitted 3 June, 2020; originally announced June 2020.

Comments: 7 pages, 2017 Irish Signal and Systems conference proceedings

Mandate-driven Networking Eco-system: A Paradigm Shift in End-to-End Communications

Authors: Ingrid Moerman, Djamal Zeghlache, Adnan Shahid, Joao F. Santos, Luiz A. DaSilva, Klaus David, John Farserotu, Ad de Ridder, Wei Liu, Jeroen Hoebeke

Abstract: The wireless industry is driven by key stakeholders that follow a holistic approach of "one-system-fits-all" that leads to moving network functionality of meeting stringent E2E communication requirements towards the core and cloud infrastructures. This trend is limiting smaller and new players for bringing in new and novel solutions. For meeting these E2E requirements, tenants and end-users need t… ▽ More The wireless industry is driven by key stakeholders that follow a holistic approach of "one-system-fits-all" that leads to moving network functionality of meeting stringent E2E communication requirements towards the core and cloud infrastructures. This trend is limiting smaller and new players for bringing in new and novel solutions. For meeting these E2E requirements, tenants and end-users need to be active players for bringing their needs and innovations. Driving E2E communication not only in terms of QoS but also overall carbon footprint and spectrum efficiency from one specific community may lead to undesirable simplifications and a higher level of abstraction of other network segments may lead to sub-optimal operations. Based on this, the paper presents a paradigm shift that will enlarge the role of wireless innovation at academia, SME's, industries and start-ups while taking into account decentralized mandate-driven intelligence in E2E communications △ Less

Submitted 6 March, 2020; v1 submitted 26 February, 2020; originally announced February 2020.

Comments: internal organisation policy

Gillian: Compositional Symbolic Execution for All

Authors: José Fragoso Santos, Petar Maksimović, Sacha-Élie Ayoun, Philippa Gardner

Abstract: We present Gillian, a language-independent framework for the development of compositional symbolic analysis tools. Gillian supports three flavours of analysis: whole-program symbolic testing, full verification, and bi-abduction. It comes with fully parametric meta-theoretical results and a modular implementation, designed to minimise the instantiation effort required of the user. We evaluate Gilli… ▽ More We present Gillian, a language-independent framework for the development of compositional symbolic analysis tools. Gillian supports three flavours of analysis: whole-program symbolic testing, full verification, and bi-abduction. It comes with fully parametric meta-theoretical results and a modular implementation, designed to minimise the instantiation effort required of the user. We evaluate Gillian by instantiating it to JavaScript and C, and perform its analyses on a set of data-structure libraries, obtaining results that indicate that Gillian is robust enough to reason about real-world programming languages. △ Less

Submitted 14 January, 2020; originally announced January 2020.

Representation Mixing for TTS Synthesis

Authors: Kyle Kastner, João Felipe Santos, Yoshua Bengio, Aaron Courville

Abstract: Recent character and phoneme-based parametric TTS systems using deep learning have shown strong performance in natural speech generation. However, the choice between character or phoneme input can create serious limitations for practical deployment, as direct control of pronunciation is crucial in certain cases. We demonstrate a simple method for combining multiple types of linguistic information… ▽ More Recent character and phoneme-based parametric TTS systems using deep learning have shown strong performance in natural speech generation. However, the choice between character or phoneme input can create serious limitations for practical deployment, as direct control of pronunciation is crucial in certain cases. We demonstrate a simple method for combining multiple types of linguistic information in a single encoder, named representation mixing, enabling flexible choice between character, phoneme, or mixed representations during inference. Experiments and user studies on a public audiobook corpus show the efficacy of our approach. △ Less

Submitted 24 November, 2018; v1 submitted 17 November, 2018; originally announced November 2018.

Comments: 5 pages, 3 figures

Speech Dereverberation with Context-aware Recurrent Neural Networks

Authors: Joao Felipe Santos, Tiago H. Falk

Abstract: In this paper, we propose a model to perform speech dereverberation by estimating its spectral magnitude from the reverberant counterpart. Our models are capable of extracting features that take into account both short and long-term dependencies in the signal through a convolutional encoder (which extracts features from a short, bounded context of frames) and a recurrent neural network for extract… ▽ More In this paper, we propose a model to perform speech dereverberation by estimating its spectral magnitude from the reverberant counterpart. Our models are capable of extracting features that take into account both short and long-term dependencies in the signal through a convolutional encoder (which extracts features from a short, bounded context of frames) and a recurrent neural network for extracting long-term information. Our model outperforms a recently proposed model that uses different context information depending on the reverberation time, without requiring any sort of additional input, yielding improvements of up to 0.4 on PESQ, 0.3 on STOI, and 1.0 on POLQA relative to reverberant speech. We also show our model is able to generalize to real room impulse responses even when only trained with simulated room impulse responses, different speakers, and high reverberation times. Lastly, listening tests show the proposed method outperforming benchmark models in reduction of perceived reverberation. △ Less

Submitted 16 November, 2017; originally announced November 2017.

Comments: Submitted to IEEE/ACM Transactions on Audio, Speech, and Language Processing

Monaural Singing Voice Separation with Skip-Filtering Connections and Recurrent Inference of Time-Frequency Mask

Authors: Stylianos Ioannis Mimilakis, Konstantinos Drossos, João F. Santos, Gerald Schuller, Tuomas Virtanen, Yoshua Bengio

Abstract: Singing voice separation based on deep learning relies on the usage of time-frequency masking. In many cases the masking process is not a learnable function or is not encapsulated into the deep learning optimization. Consequently, most of the existing methods rely on a post processing step using the generalized Wiener filtering. This work proposes a method that learns and optimizes (during trainin… ▽ More Singing voice separation based on deep learning relies on the usage of time-frequency masking. In many cases the masking process is not a learnable function or is not encapsulated into the deep learning optimization. Consequently, most of the existing methods rely on a post processing step using the generalized Wiener filtering. This work proposes a method that learns and optimizes (during training) a source-dependent mask and does not need the aforementioned post processing step. We introduce a recurrent inference algorithm, a sparse transformation step to improve the mask generation process, and a learned denoising filter. Obtained results show an increase of 0.49 dB for the signal to distortion ratio and 0.30 dB for the signal to interference ratio, compared to previous state-of-the-art approaches for monaural singing voice separation. △ Less

Submitted 13 February, 2018; v1 submitted 4 November, 2017; originally announced November 2017.

Deep Complex Networks

Authors: Chiheb Trabelsi, Olexa Bilaniuk, Ying Zhang, Dmitriy Serdyuk, Sandeep Subramanian, João Felipe Santos, Soroush Mehri, Negar Rostamzadeh, Yoshua Bengio, Christopher J Pal

Abstract: At present, the vast majority of building blocks, techniques, and architectures for deep learning are based on real-valued operations and representations. However, recent work on recurrent neural networks and older fundamental theoretical analysis suggests that complex numbers could have a richer representational capacity and could also facilitate noise-robust memory retrieval mechanisms. Despite… ▽ More At present, the vast majority of building blocks, techniques, and architectures for deep learning are based on real-valued operations and representations. However, recent work on recurrent neural networks and older fundamental theoretical analysis suggests that complex numbers could have a richer representational capacity and could also facilitate noise-robust memory retrieval mechanisms. Despite their attractive properties and potential for opening up entirely new neural architectures, complex-valued deep neural networks have been marginalized due to the absence of the building blocks required to design such models. In this work, we provide the key atomic components for complex-valued deep neural networks and apply them to convolutional feed-forward networks and convolutional LSTMs. More precisely, we rely on complex convolutions and present algorithms for complex batch-normalization, complex weight initialization strategies for complex-valued neural nets and we use them in experiments with end-to-end training schemes. We demonstrate that such complex-valued models are competitive with their real-valued counterparts. We test deep complex models on several computer vision tasks, on music transcription using the MusicNet dataset and on Speech Spectrum Prediction using the TIMIT dataset. We achieve state-of-the-art performance on these audio-related tasks. △ Less

Submitted 25 February, 2018; v1 submitted 27 May, 2017; originally announced May 2017.

Music transcription modelling and composition using deep learning

Authors: Bob L. Sturm, João Felipe Santos, Oded Ben-Tal, Iryna Korshunova

Abstract: We apply deep learning methods, specifically long short-term memory (LSTM) networks, to music transcription modelling and composition. We build and train LSTM networks using approximately 23,000 music transcriptions expressed with a high-level vocabulary (ABC notation), and use them to generate new transcriptions. Our practical aim is to create music transcription models useful in particular conte… ▽ More We apply deep learning methods, specifically long short-term memory (LSTM) networks, to music transcription modelling and composition. We build and train LSTM networks using approximately 23,000 music transcriptions expressed with a high-level vocabulary (ABC notation), and use them to generate new transcriptions. Our practical aim is to create music transcription models useful in particular contexts of music composition. We present results from three perspectives: 1) at the population level, comparing descriptive statistics of the set of training transcriptions and generated transcriptions; 2) at the individual level, examining how a generated transcription reflects the conventions of a music practice in the training transcriptions (Celtic folk); 3) at the application level, using the system for idea generation in music composition. We make our datasets, software and sound examples open and available: \url{https://github.com/IraKorshunova/folk-rnn}. △ Less

Submitted 29 April, 2016; originally announced April 2016.

Comments: 16 pages, 4 figures, contribution to 1st Conference on Computer Simulation of Musical Creativity

SRMR variants for improved blind room acoustics characterization

Authors: M. Senoussaoui, J. F. Santos, T. H. Falk

Abstract: Reverberation, especially in large rooms, severely degrades speech recognition performance and speech intelligibility. Since direct measurement of room characteristics is usually not possible, blind estimation of reverberation-related metrics such as the reverberation time (RT) and the direct-to-reverberant energy ratio (DRR) can be valuable information to speech recognition and enhancement algori… ▽ More Reverberation, especially in large rooms, severely degrades speech recognition performance and speech intelligibility. Since direct measurement of room characteristics is usually not possible, blind estimation of reverberation-related metrics such as the reverberation time (RT) and the direct-to-reverberant energy ratio (DRR) can be valuable information to speech recognition and enhancement algorithms operating in enclosed environments. The objective of this work is to evaluate the performance of five variants of blind RT and DRR estimators based on a modulation spectrum representation of reverberant speech with single- and multi-channel speech data. These models are all based on variants of the so-called Speech-to-Reverberation Modulation Energy Ratio (SRMR). We show that these measures outperform a state-of-the-art baseline based on maximum-likelihood estimation of sound decay rates in terms of root-mean square error (RMSE), as well as Pearson correlation. Compared to the baseline, the best proposed measure, called NSRMR_k , achieves a 23% relative improvement in terms of RMSE and allows for relative correlation improvements ranging from 13% to 47% for RT prediction. △ Less

Submitted 15 October, 2015; originally announced October 2015.

Comments: In Proceedings of the ACE Chal- lenge Workshop - a satellite event of IEEE-WASPAA 2015 (arXiv:1510.00383)

Report number: ACEChallenge/2015/07