-
Supervised Learning in Temporally-Coded Spiking Neural Networks with Approximate Backpropagation
Authors:
Andrew Stephan,
Brian Gardner,
Steven J. Koester,
Andre Gruning
Abstract:
In this work we propose a new supervised learning method for temporally-encoded multilayer spiking networks to perform classification. The method employs a reinforcement signal that mimics backpropagation but is far less computationally intensive. The weight update calculation at each layer requires only local data apart from this signal. We also employ a rule capable of producing specific output…
▽ More
In this work we propose a new supervised learning method for temporally-encoded multilayer spiking networks to perform classification. The method employs a reinforcement signal that mimics backpropagation but is far less computationally intensive. The weight update calculation at each layer requires only local data apart from this signal. We also employ a rule capable of producing specific output spike trains; by setting the target spike time equal to the actual spike time with a slight negative offset for key high-value neurons the actual spike time becomes as early as possible. In simulated MNIST handwritten digit classification, two-layer networks trained with this rule matched the performance of a comparable backpropagation based non-spiking network.
△ Less
Submitted 26 July, 2020;
originally announced July 2020.
-
Spin-Hall MTJ Cells for Intra-Column Competition in Hierarchical Temporal Memory
Authors:
Andrew W. Stephan,
Steven J. Koester
Abstract:
We propose a dedicated winner-take-all circuit to efficiently implement the intra-column competition between cells in Hierarchical Temporal Memory which is a crucial part of the algorithm. All inputs and outputs are charge-based for compatibility with standard CMOS. The circuit incorporates memristors for competitive advantage to emulate a column with a cell in a predictive state. The circuit can…
▽ More
We propose a dedicated winner-take-all circuit to efficiently implement the intra-column competition between cells in Hierarchical Temporal Memory which is a crucial part of the algorithm. All inputs and outputs are charge-based for compatibility with standard CMOS. The circuit incorporates memristors for competitive advantage to emulate a column with a cell in a predictive state. The circuit can also detect columns 'bursting' by passive averaging and comparison of the cell outputs. The proposed spintronic devices and circuit are thoroughly described and a series of simulations are used to predict the performance. The simulations indicate that the circuit can complete a nine-cell, nine-input competition operation in under 15 ns at a cost of about 25 pJ.
△ Less
Submitted 16 July, 2020;
originally announced July 2020.
-
SHE-MTJ Circuits for Convolutional Neural Networks
Authors:
Andrew W. Stephan,
Steven J. Koester
Abstract:
We report the performance characteristics of a notional Convolutional Neural Network based on the previously-proposed Multiply-Accumulate-Activate-Pool set, an MTJ-based spintronic circuit made to compute multiple neural functionalities in parallel. A study of image classification with the MNIST handwritten digits dataset using this network is provided via simulation. The effect of changing the we…
▽ More
We report the performance characteristics of a notional Convolutional Neural Network based on the previously-proposed Multiply-Accumulate-Activate-Pool set, an MTJ-based spintronic circuit made to compute multiple neural functionalities in parallel. A study of image classification with the MNIST handwritten digits dataset using this network is provided via simulation. The effect of changing the weight representation precision, the severity of device process variation within the MAAP sets and the computational redundancy are provided. The emulated network achieves between 90 and 95\% image classification accuracy at a cost of ~100 nJ per image.
△ Less
Submitted 16 July, 2020;
originally announced July 2020.
-
Nonvolatile Spintronic Memory Cells for Neural Networks
Authors:
Andrew W. Stephan,
Qiuwen Lou,
Michael Niemier,
X. Sharon Hu,
Steven J. Koester
Abstract:
A new spintronic nonvolatile memory cell analogous to 1T DRAM with non-destructive read is proposed. The cells can be used as neural computing units. A dual-circuit neural network architecture is proposed to leverage these devices against the complex operations involved in convolutional networks. Simulations based on HSPICE and Matlab were performed to study the performance of this architecture wh…
▽ More
A new spintronic nonvolatile memory cell analogous to 1T DRAM with non-destructive read is proposed. The cells can be used as neural computing units. A dual-circuit neural network architecture is proposed to leverage these devices against the complex operations involved in convolutional networks. Simulations based on HSPICE and Matlab were performed to study the performance of this architecture when classifying images as well as the effect of varying the size and stability of the nanomagnets. The spintronic cells outperform a purely charge-based implementation of the same network, consuming about 100 pJ total per image processed.
△ Less
Submitted 29 May, 2019;
originally announced May 2019.
-
Convolutional Neural Networks Utilizing Multifunctional Spin-Hall MTJ Neurons
Authors:
Andrew W. Stephan,
Steven J. Koester
Abstract:
We propose a new network architecture for standard spin-Hall magnetic tunnel junction-based spintronic neurons that allows them to compute multiple critical convolutional neural network functionalities simultaneously and in parallel, saving space and time. An approximation to the Rectified Linear Unit transfer function and the local pooling function are computed simultaneously with the convolution…
▽ More
We propose a new network architecture for standard spin-Hall magnetic tunnel junction-based spintronic neurons that allows them to compute multiple critical convolutional neural network functionalities simultaneously and in parallel, saving space and time. An approximation to the Rectified Linear Unit transfer function and the local pooling function are computed simultaneously with the convolution operation itself. A proof-of-concept simulation is performed on the MNIST dataset, achieving up to 98% accuracy at a cost of less than 1 nJ for all convolution, activation and pooling operations combined. The simulations are remarkably robust to thermal noise, performing well even with very small magnetic layers.
△ Less
Submitted 9 May, 2019;
originally announced May 2019.
-
Benchmarking Inverse Rashba-Edelstein Magnetoelectric Devices for Neuromorphic Computing
Authors:
Andrew W. Stephan,
Jiaxi Hu,
Steven J. Koester
Abstract:
We propose a new design for a cellular neural network with spintronic neurons and CMOS-based synapses. Harnessing the magnetoelectric and inverse Rashba-Edelstein effects allows natural emulation of the behavior of an ideal cellular network. This combination of effects offers an increase in speed and efficiency over other spintronic neural networks. A rigorous performance analysis via simulation i…
▽ More
We propose a new design for a cellular neural network with spintronic neurons and CMOS-based synapses. Harnessing the magnetoelectric and inverse Rashba-Edelstein effects allows natural emulation of the behavior of an ideal cellular network. This combination of effects offers an increase in speed and efficiency over other spintronic neural networks. A rigorous performance analysis via simulation is provided.
△ Less
Submitted 21 November, 2018;
originally announced November 2018.