-
Roadmap to Neuromorphic Computing with Emerging Technologies
Authors:
Adnan Mehonic,
Daniele Ielmini,
Kaushik Roy,
Onur Mutlu,
Shahar Kvatinsky,
Teresa Serrano-Gotarredona,
Bernabe Linares-Barranco,
Sabina Spiga,
Sergey Savelev,
Alexander G Balanov,
Nitin Chawla,
Giuseppe Desoli,
Gerardo Malavena,
Christian Monzio Compagnoni,
Zhongrui Wang,
J Joshua Yang,
Ghazi Sarwat Syed,
Abu Sebastian,
Thomas Mikolajick,
Beatriz Noheda,
Stefan Slesazeck,
Bernard Dieny,
Tuo-Hung,
Hou,
Akhil Varri
, et al. (28 additional authors not shown)
Abstract:
The roadmap is organized into several thematic sections, outlining current computing challenges, discussing the neuromorphic computing approach, analyzing mature and currently utilized technologies, providing an overview of emerging technologies, addressing material challenges, exploring novel computing concepts, and finally examining the maturity level of emerging technologies while determining t…
▽ More
The roadmap is organized into several thematic sections, outlining current computing challenges, discussing the neuromorphic computing approach, analyzing mature and currently utilized technologies, providing an overview of emerging technologies, addressing material challenges, exploring novel computing concepts, and finally examining the maturity level of emerging technologies while determining the next essential steps for their advancement.
△ Less
Submitted 5 July, 2024; v1 submitted 2 July, 2024;
originally announced July 2024.
-
Artificial Dendritic Computation: The case for dendrites in neuromorphic circuits
Authors:
Daniel John Mannion,
Anthony Joseph Kenyon
Abstract:
Bio-inspired computing has focused on neuron and synapses with great success. However, the connections between these, the dendrites, also play an important role. In this paper, we investigate the motivation for replicating dendritic computation and present a framework to guide future attempts in their construction. The framework identifies key properties of the dendrites and presents and example o…
▽ More
Bio-inspired computing has focused on neuron and synapses with great success. However, the connections between these, the dendrites, also play an important role. In this paper, we investigate the motivation for replicating dendritic computation and present a framework to guide future attempts in their construction. The framework identifies key properties of the dendrites and presents and example of dendritic computation in the task of sound localisation. We evaluate the impact of dendrites on an BiLSTM neural network's performance, finding that dendrite pre-processing reduce the size of network required for a threshold performance.
△ Less
Submitted 5 April, 2023; v1 submitted 3 April, 2023;
originally announced April 2023.
-
Memristive, Spintronic, and 2D-Materials-Based Devices to Improve and Complement Computing Hardware
Authors:
Dovydas Joksas,
AbdulAziz AlMutairi,
Oscar Lee,
Murat Cubukcu,
Antonio Lombardo,
Hidekazu Kurebayashi,
Anthony J. Kenyon,
Adnan Mehonic
Abstract:
In a data-driven economy, virtually all industries benefit from advances in information technology -- powerful computing systems are critically important for rapid technological progress. However, this progress might be at risk of slowing down if we do not address the discrepancy between our current computing power demands and what the existing technologies can offer. Key limitations to improving…
▽ More
In a data-driven economy, virtually all industries benefit from advances in information technology -- powerful computing systems are critically important for rapid technological progress. However, this progress might be at risk of slowing down if we do not address the discrepancy between our current computing power demands and what the existing technologies can offer. Key limitations to improving energy efficiency are the excessive growth of data transfer costs associated with the von Neumann architecture and the fundamental limits of complementary metal-oxide-semiconductor (CMOS) technologies, such as transistors. In this perspective article, we discuss three technologies that will likely play an essential role in future computing systems: memristive electronics, spintronics, and electronics based on 2D materials. We present how these may transform conventional digital computers and contribute to the adoption of new paradigms, like neuromorphic computing.
△ Less
Submitted 1 July, 2022; v1 submitted 11 March, 2022;
originally announced March 2022.
-
Nonideality-Aware Training for Accurate and Robust Low-Power Memristive Neural Networks
Authors:
Dovydas Joksas,
Erwei Wang,
Nikolaos Barmpatsalos,
Wing H. Ng,
Anthony J. Kenyon,
George A. Constantinides,
Adnan Mehonic
Abstract:
Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever-increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor-based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy eff…
▽ More
Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever-increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor-based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy efficiency. However, memristor nonidealities can lead to the degradation of neural network accuracy, while the attempts to mitigate these negative effects often introduce design trade-offs, such as those between power and reliability. In this work, we design nonideality-aware training of memristor-based neural networks capable of dealing with the most common device nonidealities. We demonstrate the feasibility of using high-resistance devices that exhibit high $I$-$V$ nonlinearity -- by analyzing experimental data and employing nonideality-aware training, we estimate that the energy efficiency of memristive vector-matrix multipliers is improved by three orders of magnitude ($0.715\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$ to $381\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$) while maintaining similar accuracy. We show that associating the parameters of neural networks with individual memristors allows to bias these devices towards less conductive states through regularization of the corresponding optimization problem, while modifying the validation procedure leads to more reliable estimates of performance. We demonstrate the universality and robustness of our approach when dealing with a wide range of nonidealities.
△ Less
Submitted 5 May, 2022; v1 submitted 13 December, 2021;
originally announced December 2021.
-
Brain-inspired computing: We need a master plan
Authors:
Adnan Mehonic,
Anthony J Kenyon
Abstract:
New computing technologies inspired by the brain promise fundamentally different ways to process information with extreme energy efficiency and the ability to handle the avalanche of unstructured and noisy data that we are generating at an ever-increasing rate. To realise this promise requires a brave and coordinated plan to bring together disparate research communities and to provide them with th…
▽ More
New computing technologies inspired by the brain promise fundamentally different ways to process information with extreme energy efficiency and the ability to handle the avalanche of unstructured and noisy data that we are generating at an ever-increasing rate. To realise this promise requires a brave and coordinated plan to bring together disparate research communities and to provide them with the funding, focus and support needed. We have done this in the past with digital technologies; we are in the process of doing it with quantum technologies; can we now do it for brain-inspired computing?
△ Less
Submitted 29 April, 2021;
originally announced April 2021.
-
Memristors -- from In-memory computing, Deep Learning Acceleration, Spiking Neural Networks, to the Future of Neuromorphic and Bio-inspired Computing
Authors:
Adnan Mehonic,
Abu Sebastian,
Bipin Rajendran,
Osvaldo Simeone,
Eleni Vasilaki,
Anthony J. Kenyon
Abstract:
Machine learning, particularly in the form of deep learning, has driven most of the recent fundamental developments in artificial intelligence. Deep learning is based on computational models that are, to a certain extent, bio-inspired, as they rely on networks of connected simple computing units operating in parallel. Deep learning has been successfully applied in areas such as object/pattern reco…
▽ More
Machine learning, particularly in the form of deep learning, has driven most of the recent fundamental developments in artificial intelligence. Deep learning is based on computational models that are, to a certain extent, bio-inspired, as they rely on networks of connected simple computing units operating in parallel. Deep learning has been successfully applied in areas such as object/pattern recognition, speech and natural language processing, self-driving vehicles, intelligent self-diagnostics tools, autonomous robots, knowledgeable personal assistants, and monitoring. These successes have been mostly supported by three factors: availability of vast amounts of data, continuous growth in computing power, and algorithmic innovations. The approaching demise of Moore's law, and the consequent expected modest improvements in computing power that can be achieved by scaling, raise the question of whether the described progress will be slowed or halted due to hardware limitations. This paper reviews the case for a novel beyond CMOS hardware technology, memristors, as a potential solution for the implementation of power-efficient in-memory computing, deep learning accelerators, and spiking neural networks. Central themes are the reliance on non-von-Neumann computing architectures and the need for developing tailored learning and inference algorithms. To argue that lessons from biology can be useful in providing directions for further progress in artificial intelligence, we briefly discuss an example based reservoir computing. We conclude the review by speculating on the big picture view of future neuromorphic and brain-inspired computing systems.
△ Less
Submitted 30 April, 2020;
originally announced April 2020.
-
Committee machines -- a universal method to deal with non-idealities in memristor-based neural networks
Authors:
D. Joksas,
P. Freitas,
Z. Chai,
W. H. Ng,
M. Buckwell,
C. Li,
W. D. Zhang,
Q. Xia,
A. J. Kenyon,
A. Mehonic
Abstract:
Artificial neural networks are notoriously power- and time-consuming when implemented on conventional von Neumann computing systems. Consequently, recent years have seen an emergence of research in machine learning hardware that strives to bring memory and computing closer together. A popular approach is to realise artificial neural networks in hardware by implementing their synaptic weights using…
▽ More
Artificial neural networks are notoriously power- and time-consuming when implemented on conventional von Neumann computing systems. Consequently, recent years have seen an emergence of research in machine learning hardware that strives to bring memory and computing closer together. A popular approach is to realise artificial neural networks in hardware by implementing their synaptic weights using memristive devices. However, various device- and system-level non-idealities usually prevent these physical implementations from achieving high inference accuracy. We suggest applying a well-known concept in computer science -- committee machines -- in the context of memristor-based neural networks. Using simulations and experimental data from three different types of memristive devices, we show that committee machines employing ensemble averaging can successfully increase inference accuracy in physically implemented neural networks that suffer from faulty devices, device-to-device variability, random telegraph noise and line resistance. Importantly, we demonstrate that the accuracy can be improved even without increasing the total number of memristors.
△ Less
Submitted 27 August, 2020; v1 submitted 14 September, 2019;
originally announced September 2019.
