High-Accuracy Inference in Neuromorphic Circuits using Hardware-Aware Training
Authors:
Borna Obradovic,
Titash Rakshit,
Ryan Hatcher,
Jorge A. Kittl,
Mark S. Rodder
Abstract:
Neuromorphic Multiply-And-Accumulate (MAC) circuits utilizing synaptic weight elements based on SRAM or novel Non-Volatile Memories (NVMs) provide a promising approach for highly efficient hardware representations of neural networks. NVM density and robustness requirements suggest that off-line training is the right choice for "edge" devices, since the requirements for synapse precision are much l…
▽ More
Neuromorphic Multiply-And-Accumulate (MAC) circuits utilizing synaptic weight elements based on SRAM or novel Non-Volatile Memories (NVMs) provide a promising approach for highly efficient hardware representations of neural networks. NVM density and robustness requirements suggest that off-line training is the right choice for "edge" devices, since the requirements for synapse precision are much less stringent. However, off-line training using ideal mathematical weights and activations can result in significant loss of inference accuracy when applied to non-ideal hardware. Non-idealities such as multi-bit quantization of weights and activations, non-linearity of weights, finite max/min ratios of NVM elements, and asymmetry of positive and negative weight components all result in degraded inference accuracy. In this work, it is demonstrated that non-ideal Multi-Layer Perceptron (MLP) architectures using low bitwidth weights and activations can be trained with negligible loss of inference accuracy relative to their Floating Point-trained counterparts using a proposed off-line, continuously differentiable HW-aware training algorithm. The proposed algorithm is applicable to a wide range of hardware models, and uses only standard neural network training methods. The algorithm is demonstrated on the MNIST and EMNIST datasets, using standard MLPs.
△ Less
Submitted 13 September, 2018;
originally announced September 2018.
A Multi-Bit Neuromorphic Weight Cell using Ferroelectric FETs, suitable for SoC Integration
Authors:
Borna Obradovic,
Titash Rakshit,
Ryan Hatcher,
Jorge Kittl,
Rwik Sengupta,
Joon Goo Hong,
Mark S. Rodder
Abstract:
A multi-bit digital weight cell for high-performance, inference-only non-GPU-like neuromorphic accelerators is presented. The cell is designed with simplicity of peripheral circuitry in mind. Non-volatile storage of weights which eliminates the need for DRAM access is based on FeFETs and is purely digital. The Multiply-and-Accumulate operation is performed using passive resistors, gated by FeFETs.…
▽ More
A multi-bit digital weight cell for high-performance, inference-only non-GPU-like neuromorphic accelerators is presented. The cell is designed with simplicity of peripheral circuitry in mind. Non-volatile storage of weights which eliminates the need for DRAM access is based on FeFETs and is purely digital. The Multiply-and-Accumulate operation is performed using passive resistors, gated by FeFETs. The resulting weight cell offers a high degree of linearity and a large ON/OFF ratio. The key performance tradeoffs are investigated, and the device requirements are elucidated.
△ Less
Submitted 22 October, 2017;
originally announced October 2017.
