Showing 1–22 of 22 results for author: Qiao, N

High-Performance Temporal Reversible Spiking Neural Networks with $O(L)$ Training Memory and $O(1)$ Inference Cost

Authors: JiaKui Hu, Man Yao, Xuerui Qiu, Yuhong Chou, Yuxuan Cai, Ning Qiao, Yonghong Tian, Bo XU, Guoqi Li

Abstract: Multi-timestep simulation of brain-inspired Spiking Neural Networks (SNNs) boost memory requirements during training and increase inference energy cost. Current training methods cannot simultaneously solve both training and inference dilemmas. This work proposes a novel Temporal Reversible architecture for SNNs (T-RevSNN) to jointly address the training and inference challenges by altering the for… ▽ More Multi-timestep simulation of brain-inspired Spiking Neural Networks (SNNs) boost memory requirements during training and increase inference energy cost. Current training methods cannot simultaneously solve both training and inference dilemmas. This work proposes a novel Temporal Reversible architecture for SNNs (T-RevSNN) to jointly address the training and inference challenges by altering the forward propagation of SNNs. We turn off the temporal dynamics of most spiking neurons and design multi-level temporal reversible interactions at temporal turn-on spiking neurons, resulting in a $O(L)$ training memory. Combined with the temporal reversible nature, we redesign the input encoding and network organization of SNNs to achieve $O(1)$ inference energy cost. Then, we finely adjust the internal units and residual connections of the basic SNN block to ensure the effectiveness of sparse temporal information interaction. T-RevSNN achieves excellent accuracy on ImageNet, while the memory efficiency, training time acceleration, and inference energy efficiency can be significantly improved by $8.6 \times$, $2.0 \times$, and $1.6 \times$, respectively. This work is expected to break the technical bottleneck of significantly increasing memory cost and training time for large-scale SNNs while maintaining high performance and low inference energy cost. Source code and models are available at: https://github.com/BICLab/T-RevSNN. △ Less

Submitted 26 May, 2024; originally announced May 2024.

Comments: Accepted by ICML2024

Vector-Symbolic Architecture for Event-Based Optical Flow

Authors: Hongzhi You, Yijun Cao, Wei Yuan, Fanjun Wang, Ning Qiao, Yongjie Li

Abstract: From a perspective of feature matching, optical flow estimation for event cameras involves identifying event correspondences by comparing feature similarity across accompanying event frames. In this work, we introduces an effective and robust high-dimensional (HD) feature descriptor for event frames, utilizing Vector Symbolic Architectures (VSA). The topological similarity among neighboring variab… ▽ More From a perspective of feature matching, optical flow estimation for event cameras involves identifying event correspondences by comparing feature similarity across accompanying event frames. In this work, we introduces an effective and robust high-dimensional (HD) feature descriptor for event frames, utilizing Vector Symbolic Architectures (VSA). The topological similarity among neighboring variables within VSA contributes to the enhanced representation similarity of feature descriptors for flow-matching points, while its structured symbolic representation capacity facilitates feature fusion from both event polarities and multiple spatial scales. Based on this HD feature descriptor, we propose a novel feature matching framework for event-based optical flow, encompassing both model-based (VSA-Flow) and self-supervised learning (VSA-SM) methods. In VSA-Flow, accurate optical flow estimation validates the effectiveness of HD feature descriptors. In VSA-SM, a novel similarity maximization method based on the HD feature descriptor is proposed to learn optical flow in a self-supervised way from events alone, eliminating the need for auxiliary grayscale images. Evaluation results demonstrate that our VSA-based method achieves superior accuracy in comparison to both model-based and self-supervised learning methods on the DSEC benchmark, while remains competitive among both methods on the MVSEC benchmark. This contribution marks a significant advancement in event-based optical flow within the feature matching methodology. △ Less

Submitted 15 May, 2024; v1 submitted 13 May, 2024; originally announced May 2024.

Single Image Super-Resolution Based on Global-Local Information Synergy

Authors: Nianzu Qiao, Lamei Di, Changyin Sun

Abstract: Although several image super-resolution solutions exist, they still face many challenges. CNN-based algorithms, despite the reduction in computational complexity, still need to improve their accuracy. While Transformer-based algorithms have higher accuracy, their ultra-high computational complexity makes them difficult to be accepted in practical applications. To overcome the existing challenges,… ▽ More Although several image super-resolution solutions exist, they still face many challenges. CNN-based algorithms, despite the reduction in computational complexity, still need to improve their accuracy. While Transformer-based algorithms have higher accuracy, their ultra-high computational complexity makes them difficult to be accepted in practical applications. To overcome the existing challenges, a novel super-resolution reconstruction algorithm is proposed in this paper. The algorithm achieves a significant increase in accuracy through a unique design while maintaining a low complexity. The core of the algorithm lies in its cleverly designed Global-Local Information Extraction Module and Basic Block Module. By combining global and local information, the Global-Local Information Extraction Module aims to understand the image content more comprehensively so as to recover the global structure and local details in the image more accurately, which provides rich information support for the subsequent reconstruction process. Experimental results show that the comprehensive performance of the algorithm proposed in this paper is optimal, providing an efficient and practical new solution in the field of super-resolution reconstruction. △ Less

Submitted 2 May, 2024; originally announced May 2024.

MCMS: Multi-Category Information and Multi-Scale Stripe Attention for Blind Motion Deblurring

Authors: Nianzu Qiao, Lamei Di, Changyin Sun

Abstract: Deep learning-based motion deblurring techniques have advanced significantly in recent years. This class of techniques, however, does not carefully examine the inherent flaws in blurry images. For instance, low edge and structural information are traits of blurry images. The high-frequency component of blurry images is edge information, and the low-frequency component is structure information. A b… ▽ More Deep learning-based motion deblurring techniques have advanced significantly in recent years. This class of techniques, however, does not carefully examine the inherent flaws in blurry images. For instance, low edge and structural information are traits of blurry images. The high-frequency component of blurry images is edge information, and the low-frequency component is structure information. A blind motion deblurring network (MCMS) based on multi-category information and multi-scale stripe attention mechanism is proposed. Given the respective characteristics of the high-frequency and low-frequency components, a three-stage encoder-decoder model is designed. Specifically, the first stage focuses on extracting the features of the high-frequency component, the second stage concentrates on extracting the features of the low-frequency component, and the third stage integrates the extracted low-frequency component features, the extracted high-frequency component features, and the original blurred image in order to recover the final clear image. As a result, the model effectively improves motion deblurring by fusing the edge information of the high-frequency component and the structural information of the low-frequency component. In addition, a grouped feature fusion technique is developed so as to achieve richer, more three-dimensional and comprehensive utilization of various types of features at a deep level. Next, a multi-scale stripe attention mechanism (MSSA) is designed, which effectively combines the anisotropy and multi-scale information of the image, a move that significantly enhances the capability of the deep model in feature representation. Large-scale comparative studies on various datasets show that the strategy in this paper works better than the recently published measures. △ Less

Submitted 2 May, 2024; originally announced May 2024.

Quantum-Inspired Machine Learning for Molecular Docking

Authors: Runqiu Shu, Bowen Liu, Zhaoping Xiong, Xiaopeng Cui, Yunting Li, Wei Cui, Man-Hong Yung, Nan Qiao

Abstract: Molecular docking is an important tool for structure-based drug design, accelerating the efficiency of drug development. Complex and dynamic binding processes between proteins and small molecules require searching and sampling over a wide spatial range. Traditional docking by searching for possible binding sites and conformations is computationally complex and results poorly under blind docking. Q… ▽ More Molecular docking is an important tool for structure-based drug design, accelerating the efficiency of drug development. Complex and dynamic binding processes between proteins and small molecules require searching and sampling over a wide spatial range. Traditional docking by searching for possible binding sites and conformations is computationally complex and results poorly under blind docking. Quantum-inspired algorithms combining quantum properties and annealing show great advantages in solving combinatorial optimization problems. Inspired by this, we achieve an improved in blind docking by using quantum-inspired combined with gradients learned by deep learning in the encoded molecular space. Numerical simulation shows that our method outperforms traditional docking algorithms and deep learning-based algorithms over 10\%. Compared to the current state-of-the-art deep learning-based docking algorithm DiffDock, the success rate of Top-1 (RMSD<2) achieves an improvement from 33\% to 35\% in our same setup. In particular, a 6\% improvement is realized in the high-precision region(RMSD<1) on molecules data unseen in DiffDock, which demonstrates the well-generalized of our method. △ Less

Submitted 21 February, 2024; v1 submitted 22 January, 2024; originally announced January 2024.

DYNAP-SE2: a scalable multi-core dynamic neuromorphic asynchronous spiking neural network processor

Authors: Ole Richter, Chenxi Wu, Adrian M. Whatley, German Köstinger, Carsten Nielsen, Ning Qiao, Giacomo Indiveri

Abstract: With the remarkable progress that technology has made, the need for processing data near the sensors at the edge has increased dramatically. The electronic systems used in these applications must process data continuously, in real-time, and extract relevant information using the smallest possible energy budgets. A promising approach for implementing always-on processing of sensory signals that sup… ▽ More With the remarkable progress that technology has made, the need for processing data near the sensors at the edge has increased dramatically. The electronic systems used in these applications must process data continuously, in real-time, and extract relevant information using the smallest possible energy budgets. A promising approach for implementing always-on processing of sensory signals that supports on-demand, sparse, and edge-computing is to take inspiration from biological nervous system. Following this approach, we present a brain-inspired platform for prototyping real-time event-based Spiking Neural Networks (SNNs). The system proposed supports the direct emulation of dynamic and realistic neural processing phenomena such as short-term plasticity, NMDA gating, AMPA diffusion, homeostasis, spike frequency adaptation, conductance-based dendritic compartments and spike transmission delays. The analog circuits that implement such primitives are paired with a low latency asynchronous digital circuits for routing and mapping events. This asynchronous infrastructure enables the definition of different network architectures, and provides direct event-based interfaces to convert and encode data from event-based and continuous-signal sensors. Here we describe the overall system architecture, we characterize the mixed signal analog-digital circuits that emulate neural dynamics, demonstrate their features with experimental measurements, and present a low- and high-level software ecosystem that can be used for configuring the system. The flexibility to emulate different biologically plausible neural networks, and the chip's ability to monitor both population and single neuron signals in real-time, allow to develop and validate complex models of neural processing for both basic research and edge-computing applications. △ Less

Submitted 10 November, 2023; v1 submitted 30 September, 2023; originally announced October 2023.

Comments: *Ole Richter and Chenxi Wu contributed equally

Journal ref: Neuromorph. Comput. Eng. 4 (2024)

ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation

Authors: Xuefeng Hu, Ke Zhang, Lu Xia, Albert Chen, Jiajia Luo, Yuyin Sun, Ken Wang, Nan Qiao, Xiao Zeng, Min Sun, Cheng-Hao Kuo, Ram Nevatia

Abstract: Large-scale Pre-Training Vision-Language Model such as CLIP has demonstrated outstanding performance in zero-shot classification, e.g. achieving 76.3% top-1 accuracy on ImageNet without seeing any example, which leads to potential benefits to many tasks that have no labeled data. However, while applying CLIP to a downstream target domain, the presence of visual and text domain gaps and cross-modal… ▽ More Large-scale Pre-Training Vision-Language Model such as CLIP has demonstrated outstanding performance in zero-shot classification, e.g. achieving 76.3% top-1 accuracy on ImageNet without seeing any example, which leads to potential benefits to many tasks that have no labeled data. However, while applying CLIP to a downstream target domain, the presence of visual and text domain gaps and cross-modality misalignment can greatly impact the model performance. To address such challenges, we propose ReCLIP, the first source-free domain adaptation method for vision-language models, which does not require any source data or target labeled data. ReCLIP first learns a projection space to mitigate the misaligned visual-text embeddings and learns pseudo labels, and then deploys cross-modality self-training with the pseudo labels, to update visual and text encoders, refine labels and reduce domain gaps and misalignments iteratively. With extensive experiments, we demonstrate ReCLIP reduces the average error rate of CLIP from 30.17% to 25.06% on 22 image classification benchmarks. Code available at https://github.com/michiganleon/ReCLIP_WACV. △ Less

Submitted 13 December, 2023; v1 submitted 4 August, 2023; originally announced August 2023.

Comments: Accepted as Oral Paper by 2024 IEEE CVF Winter Conference on Applications of Computer Vision (WACV)

Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline

Authors: Ole Richter, Yannan Xing, Michele De Marchi, Carsten Nielsen, Merkourios Katsimpris, Roberto Cattaneo, Yudi Ren, Yalun Hu, Qian Liu, Sadique Sheik, Tugba Demirci, Ning Qiao

Abstract: Edge computing solutions that enable the extraction of high-level information from a variety of sensors is in increasingly high demand. This is due to the increasing number of smart devices that require sensory processing for their application on the edge. To tackle this problem, we present a smart vision sensor System on Chip (SoC), featuring an event-based camera and a low-power asynchronous spi… ▽ More Edge computing solutions that enable the extraction of high-level information from a variety of sensors is in increasingly high demand. This is due to the increasing number of smart devices that require sensory processing for their application on the edge. To tackle this problem, we present a smart vision sensor System on Chip (SoC), featuring an event-based camera and a low-power asynchronous spiking Convolutional Neural Network (sCNN) computing architecture embedded on a single chip. By combining both sensor and processing on a single die, we can lower unit production costs significantly. Moreover, the simple end-to-end nature of the SoC facilitates small stand-alone applications as well as functioning as an edge node in larger systems. The event-driven nature of the vision sensor delivers high-speed signals in a sparse data stream. This is reflected in the processing pipeline, which focuses on optimising highly sparse computation and minimising latency for 9 sCNN layers to 3.36μs for an incoming event. Overall, this results in an extremely low-latency visual processing pipeline deployed on a small form factor with a low energy budget and sensor cost. We present the asynchronous architecture, the individual blocks, and the sCNN processing principle and benchmark against other sCNN capable processors. △ Less

Submitted 27 May, 2024; v1 submitted 13 April, 2023; originally announced April 2023.

Comments: accepted and presented at 28th IEEE International Symposium On Asynchronous Circuits and Systems (ASYNC) 2023

Journal ref: IEEE ASYNC 2023

PoPeC: PAoI-Centric Task Offloading with Priority over Unreliable Channels

Authors: Nan Qiao, Sheng Yue, Yongmin Zhang, Ju Ren

Abstract: Freshness-aware computation offloading has garnered great attention recently in the edge computing arena, with the aim of promptly obtaining up-to-date information and minimizing the transmission of outdated data. However, most of the existing work assumes that wireless channels are reliable and neglect the dynamics and stochasticity thereof. In addition, varying priorities of offloading tasks alo… ▽ More Freshness-aware computation offloading has garnered great attention recently in the edge computing arena, with the aim of promptly obtaining up-to-date information and minimizing the transmission of outdated data. However, most of the existing work assumes that wireless channels are reliable and neglect the dynamics and stochasticity thereof. In addition, varying priorities of offloading tasks along with heterogeneous computing units also pose significant challenges in effective task scheduling and resource allocation. To address these challenges, we cast the freshness-aware task offloading problem as a multi-priority optimization problem, considering the unreliability of wireless channels, the heterogeneity of edge servers, and prioritized users. Based on the nonlinear fractional programming and ADMM-Consensus method, we propose a joint resource allocation and task offloading algorithm to solve the original problem iteratively. To improve communication efficiency, we further devise a distributed asynchronous variant for the proposed algorithm. We rigorously analyze the performance and convergence of the proposed algorithms and conduct extensive simulations to corroborate their efficacy and superiority over the existing baselines. △ Less

Submitted 20 December, 2023; v1 submitted 27 March, 2023; originally announced March 2023.

Journal ref: IEEE/ACM Transactions on Networking 2024

CameraPose: Weakly-Supervised Monocular 3D Human Pose Estimation by Leveraging In-the-wild 2D Annotations

Authors: Cheng-Yen Yang, Jiajia Luo, Lu Xia, Yuyin Sun, Nan Qiao, Ke Zhang, Zhongyu Jiang, Jenq-Neng Hwang

Abstract: To improve the generalization of 3D human pose estimators, many existing deep learning based models focus on adding different augmentations to training poses. However, data augmentation techniques are limited to the "seen" pose combinations and hard to infer poses with rare "unseen" joint positions. To address this problem, we present CameraPose, a weakly-supervised framework for 3D human pose est… ▽ More To improve the generalization of 3D human pose estimators, many existing deep learning based models focus on adding different augmentations to training poses. However, data augmentation techniques are limited to the "seen" pose combinations and hard to infer poses with rare "unseen" joint positions. To address this problem, we present CameraPose, a weakly-supervised framework for 3D human pose estimation from a single image, which can not only be applied on 2D-3D pose pairs but also on 2D alone annotations. By adding a camera parameter branch, any in-the-wild 2D annotations can be fed into our pipeline to boost the training diversity and the 3D poses can be implicitly learned by reprojecting back to 2D. Moreover, CameraPose introduces a refinement network module with confidence-guided loss to further improve the quality of noisy 2D keypoints extracted by 2D pose estimators. Experimental results demonstrate that the CameraPose brings in clear improvements on cross-scenario datasets. Notably, it outperforms the baseline method by 3mm on the most challenging dataset 3DPW. In addition, by combining our proposed refinement network module with existing 3D pose estimators, their performance can be improved in cross-scenario evaluation. △ Less

Submitted 8 January, 2023; originally announced January 2023.

TODG: Distributed Task Offloading with Delay Guarantees for Edge Computing

Authors: Sheng Yue, Ju Ren, Nan Qiao, Yongmin Zhang, Hongbo Jiang, Yaoxue Zhang, Yuanyuan Yang

Abstract: Edge computing has been an efficient way to provide prompt and near-data computing services for resource-and-delay sensitive IoT applications via computation offloading. Effective computation offloading strategies need to comprehensively cope with several major issues, including the allocation of dynamic communication and computational resources, the deadline constraints of heterogeneous tasks, an… ▽ More Edge computing has been an efficient way to provide prompt and near-data computing services for resource-and-delay sensitive IoT applications via computation offloading. Effective computation offloading strategies need to comprehensively cope with several major issues, including the allocation of dynamic communication and computational resources, the deadline constraints of heterogeneous tasks, and the requirements for computationally inexpensive and distributed algorithms. However, most of the existing works mainly focus on part of these issues, which would not suffice to achieve expected performance in complex and practical scenarios. To tackle this challenge, in this paper, we systematically study a distributed computation offloading problem with hard delay constraints, where heterogeneous computational tasks require continually offloading to a set of edge servers via a limiting number of stochastic communication channels. The task offloading problem is then cast as a delay-constrained long-term stochastic optimization problem under unknown priori statistical knowledge. To resolve this problem, we first provide a technical path to transform and decompose it into several slot-level subproblems, then we develop a distributed online algorithm, namely TODG, to efficiently allocate the resources and schedule the offloading tasks with delay guarantees. Further, we present a comprehensive analysis for TODG, in terms of the optimality gap, the delay guarantees, and the impact of system parameters. Extensive simulation results demonstrate the effectiveness and efficiency of TODG. △ Less

Submitted 23 September, 2021; v1 submitted 7 January, 2021; originally announced January 2021.

Ultra-Low-Power FDSOI Neural Circuits for Extreme-Edge Neuromorphic Intelligence

Authors: Arianna Rubino, Can Livanelioglu, Ning Qiao, Melika Payvand, Giacomo Indiveri

Abstract: Recent years have seen an increasing interest in the development of artificial intelligence circuits and systems for edge computing applications. In-memory computing mixed-signal neuromorphic architectures provide promising ultra-low-power solutions for edge-computing sensory-processing applications, thanks to their ability to emulate spiking neural networks in real-time. The fine-grain parallelis… ▽ More Recent years have seen an increasing interest in the development of artificial intelligence circuits and systems for edge computing applications. In-memory computing mixed-signal neuromorphic architectures provide promising ultra-low-power solutions for edge-computing sensory-processing applications, thanks to their ability to emulate spiking neural networks in real-time. The fine-grain parallelism offered by this approach allows such neural circuits to process the sensory data efficiently by adapting their dynamics to the ones of the sensed signals, without having to resort to the time-multiplexed computing paradigm of von Neumann architectures. To reduce power consumption even further, we present a set of mixed-signal analog/digital circuits that exploit the features of advanced Fully-Depleted Silicon on Insulator (FDSOI) integration processes. Specifically, we explore the options of advanced FDSOI technologies to address analog design issues and optimize the design of the synapse integrator and of the adaptive neuron circuits accordingly. We present circuit simulation results and demonstrate the circuit's ability to produce biologically plausible neural dynamics with compact designs, optimized for the realization of large-scale spiking neural networks in neuromorphic processors. △ Less

Submitted 14 July, 2020; v1 submitted 25 June, 2020; originally announced June 2020.

Comments: 11 pages, 9 figures, TCAS submission

ACM Class: B.7; C.3

Analog circuits for mixed-signal neuromorphic computing architectures in 28 nm FD-SOI technology

Authors: Ning Qiao, Giacomo Indiveri

Abstract: Developing mixed-signal analog-digital neuromorphic circuits in advanced scaled processes poses significant design challenges. We present compact and energy efficient sub-threshold analog synapse and neuron circuits, optimized for a 28 nm FD-SOI process, to implement massively parallel large-scale neuromorphic computing systems. We describe the techniques used for maximizing density with mixed-mod… ▽ More Developing mixed-signal analog-digital neuromorphic circuits in advanced scaled processes poses significant design challenges. We present compact and energy efficient sub-threshold analog synapse and neuron circuits, optimized for a 28 nm FD-SOI process, to implement massively parallel large-scale neuromorphic computing systems. We describe the techniques used for maximizing density with mixed-mode analog/digital synaptic weight configurations, and the methods adopted for minimizing the effect of channel leakage current, in order to implement efficient analog computation based on pA-nA small currents. We present circuit simulation results, based on a new chip that has been recently taped out, to demonstrate how the circuits can be useful for both low-frequency operation in systems that need to interact with the environment in real-time, and for high-frequency operation for fast data processing in different types of spiking neural network architectures. △ Less

Submitted 18 August, 2019; originally announced August 2019.

Comments: 2017 IEEE SOI-3D-Subthreshold Microelectronics Technology Unified Conference (S3S)

A bi-directional Address-Event transceiver block for low-latency inter-chip communication in neuromorphic systems

Authors: Ning Qiao, Giacomo Indiveri

Abstract: Neuromorphic systems typically use the Address-Event Representation (AER) to transmit signals among nodes, cores, and chips. Communication of Address-Events (AEs) between neuromorphic cores/chips typically requires two parallel digital signal buses for Input/Output (I/O) operations. This requirement can become very expensive for large-scale systems in terms of both dedicated I/O pins and power con… ▽ More Neuromorphic systems typically use the Address-Event Representation (AER) to transmit signals among nodes, cores, and chips. Communication of Address-Events (AEs) between neuromorphic cores/chips typically requires two parallel digital signal buses for Input/Output (I/O) operations. This requirement can become very expensive for large-scale systems in terms of both dedicated I/O pins and power consumption. In this paper we present a compact fully asynchronous event-driven transmitter/receiver block that is both power efficient and I/O efficient. This block implements high-throughput low-latency bi-directional communication through a parallel AER bus. We show that by placing the proposed AE transceiver block in two separate chips and linking them by a single AER bus, we can drive the communication and switch the transmission direction of the shared bus on a single event basis, from either side with low-latency. We present experimental results that validate the circuits proposed and demonstrate reliable bi-directional event transmission with high-throughput. The proposed AE block, integrated in a neuromorphic chip fabricated using a 28 nm FDSOI process, occupies a silicon die area of 140 μm x 70 μm. The experimental measurements show that the event-driven AE block combined with standard digital I/Os has a direction switch latency of 5 ns and can achieve a worst-case bi-directional event transmission throughput of 28.6M Events/second while consuming 11 pJ per event (26-bit) delivery. △ Less

Submitted 18 August, 2019; originally announced August 2019.

Comments: 2018 IEEE International Symposium on Circuits and Systems (ISCAS)

Automatic gain control of ultra-low leakage synaptic scaling homeostatic plasticity circuits

Authors: Ning Qiao, Giacomo Indiveri, Chiara Bartolozzi

Abstract: Homeostatic plasticity is a stabilizing mechanism that allows neural systems to maintain their activity around a functional operating point. This is an extremely useful mechanism for neuromorphic computing systems, as it can be used to compensate for chronic shifts, for example due to changes in the network structure. However, it is important that this plasticity mechanism operates on time scales… ▽ More Homeostatic plasticity is a stabilizing mechanism that allows neural systems to maintain their activity around a functional operating point. This is an extremely useful mechanism for neuromorphic computing systems, as it can be used to compensate for chronic shifts, for example due to changes in the network structure. However, it is important that this plasticity mechanism operates on time scales that are much longer than conventional synaptic plasticity ones, in order to not interfere with the learning process. In this paper we present a novel ultra-low leakage cell and an automatic gain control scheme that can adapt the gain of analog log-domain synapse circuits over extremely long time scales. To validate the proposed scheme, we implemented the ultra-low leakage cell in a standard 180 nm Complementary Metal-Oxide-Semiconductor (CMOS) process, and integrated it in an array of dynamic synapses connected to an adaptive integrate and fire neuron. We describe the circuit and demonstrate how it can be configured to scale the gain of all synapses afferent to the silicon neuron in a way to keep the neuron's average firing rate constant around a set operating point. The circuit occupies a silicon area of 84 μm x 22 μm and consumes approximately 10.8 nW with a 1.8 V supply voltage. It exhibits time constants of up to 25 kilo-seconds, thanks to a controllable leakage current that can be scaled down to 1.2 atto-Amps (7.5 electrons/s). △ Less

Submitted 18 August, 2019; originally announced August 2019.

Comments: 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS). arXiv admin note: substantial text overlap with arXiv:1709.05633

Scaling mixed-signal neuromorphic processors to 28 nm FD-SOI technologies

Authors: Ning Qiao, Giacomo Inidveri

Abstract: As processes continue to scale aggressively, the design of deep sub-micron, mixed-signal design is becoming more and more challenging. In this paper we present an analysis of scaling multi-core mixed-signal neuromorphic processors to advanced 28 nm FD-SOI nodes. We address analog design issues which arise from the use of advanced process, including the problem of large leakage currents and device… ▽ More As processes continue to scale aggressively, the design of deep sub-micron, mixed-signal design is becoming more and more challenging. In this paper we present an analysis of scaling multi-core mixed-signal neuromorphic processors to advanced 28 nm FD-SOI nodes. We address analog design issues which arise from the use of advanced process, including the problem of large leakage currents and device mismatch, and asynchronous digital design issues. We present the outcome of Monte Carlo Analysis and circuit simulations of neuromorphic sub threshold analog/digital neuron circuits which reproduce biologically plausible responses. We describe the AER used to implement PCHB based asynchronous QDI routing processes in multi-core neuromorphic architectures and validate their operation via circuit simulation results. Finally we describe the implementation of custom 28 nm CAM based memory resources utilized in these multi-core neuromorphic processor and discuss the possibility of increasing density by using advanced RRAM devices integrated in the 28 nm Fully-Depleted Silicon on Insulator (FD-SOI) process. △ Less

Submitted 18 August, 2019; originally announced August 2019.

Comments: 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS)

An auto-scaling wide dynamic range current to frequency converter for real-time monitoring of signals in neuromorphic systems

Authors: Ning Qiao, Giacomo Indiveri

Abstract: Neuromorphic systems typically employ current-mode circuits that model neural dynamics and produce output currents that range from few pico-Amperes to hundreds of micro-Amperes. On-line real-time monitoring of the signals produced by these circuits is crucial, for prototyping and debugging purposes, as well as for analyzing and understanding the network dynamics and computational properties. To th… ▽ More Neuromorphic systems typically employ current-mode circuits that model neural dynamics and produce output currents that range from few pico-Amperes to hundreds of micro-Amperes. On-line real-time monitoring of the signals produced by these circuits is crucial, for prototyping and debugging purposes, as well as for analyzing and understanding the network dynamics and computational properties. To this end, we propose a compact on-chip auto-scaling Current to Frequency Converter (CFC) for real-time monitoring of analog currents in mixed-signal/analog neuromorphic electronic systems. The proposed CFC is a self-timed asynchronous circuit that has a wide dynamic input range of up to 6 decades, ranging from pico-Amps to micro-Amps, with high current measurement sensitivity. To produce a linear output frequency response, while properly covering the wide dynamic input range, the circuit automatically detects the scale of the input current and adjusts the scale of its output firing rate accordingly. Here we describe the proposed circuit and present experimental results measured from multiple instances of the circuit, implemented using a standard 180 nm CMOS process, and interfaced to silicon neuron and synapse circuits for real-time current monitoring. We demonstrate how the circuit is suitable for measuring neural dynamics by showing the converted response properties of the chip silicon neurons and synapses as they are stimulated by input spikes. △ Less

Submitted 18 August, 2019; originally announced August 2019.

Comments: 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS)

A clock-less ultra-low power bit-serial LVDS link for Address-Event multi-chip systems

Authors: Ning Qiao, Giacomo Indiveri

Abstract: We present a power efficient clock-less fully asynchronous bit-serial Low Voltage Differential Signaling (LVDS) link with event-driven instant wake-up and self-sleep features, optimized for high speed inter-chip communication of asynchronous address-events between neuromorphic chips. The proposed LVDS link makes use of the Level-Encoded Dual-Rail (LEDR) representation and a token-ring architecture… ▽ More We present a power efficient clock-less fully asynchronous bit-serial Low Voltage Differential Signaling (LVDS) link with event-driven instant wake-up and self-sleep features, optimized for high speed inter-chip communication of asynchronous address-events between neuromorphic chips. The proposed LVDS link makes use of the Level-Encoded Dual-Rail (LEDR) representation and a token-ring architecture to encode and transmit data, avoiding the use of conventional large ClockData Recovery (CDR) modules with power-hungry DLL or PLL circuits. We implemented the LVDS circuits in a device fabricated with a standard 0.18 um CMOS process. The total silicon area used for such block is of 0.14 mm^2. We present experimental measurement results to demonstrate that, with a bit rate of 1.5 Gbps and an event width of 32-bit, the proposed LVDS link can achieve transmission event rates of 35.7 M Events/second with current consumption of 19.3 mA and 3.57 mA for receiver and transmitter blocks, respectively. Given the clock-less and instant on/off design choices made, the power consumption of the whole link depends linearly on the data transmission rate. We show that the current consumption can go down to sub-uA for low event rates (e.g., <1k Events/second), with a floor of 80 nA for transmitter and 42 nA for receiver, determined mainly by static off-leakage currents. △ Less

Submitted 18 August, 2019; originally announced August 2019.

Comments: 24th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC 2018)

Adaptive motor control and learning in a spiking neural network realised on a mixed-signal neuromorphic processor

Authors: Sebastian Glatz, Julien N. P. Martel, Raphaela Kreiser, Ning Qiao, Yulia Sandamirskaya

Abstract: Neuromorphic computing is a new paradigm for design of both the computing hardware and algorithms inspired by biological neural networks. The event-based nature and the inherent parallelism make neuromorphic computing a promising paradigm for building efficient neural network based architectures for control of fast and agile robots. In this paper, we present a spiking neural network architecture t… ▽ More Neuromorphic computing is a new paradigm for design of both the computing hardware and algorithms inspired by biological neural networks. The event-based nature and the inherent parallelism make neuromorphic computing a promising paradigm for building efficient neural network based architectures for control of fast and agile robots. In this paper, we present a spiking neural network architecture that uses sensory feedback to control rotational velocity of a robotic vehicle. When the velocity reaches the target value, the mapping from the target velocity of the vehicle to the correct motor command, both represented in the spiking neural network on the neuromorphic device, is autonomously stored on the device using on-chip plastic synaptic weights. We validate the controller using a wheel motor of a miniature mobile vehicle and inertia measurement unit as the sensory feedback and demonstrate online learning of a simple 'inverse model' in a two-layer spiking neural network on the neuromorphic chip. The prototype neuromorphic device that features 256 spiking neurons allows us to realise a simple proof of concept architecture for the purely neuromorphic motor control and learning. The architecture can be easily scaled-up if a larger neuromorphic device is available. △ Less

Submitted 25 October, 2018; originally announced October 2018.

Comments: 6+1 pages, 4 figures, will appear in one of the Robotics conferences

Journal ref: IEEE International Conference on Robotics and Automation (ICRA) 2019

Large-Scale Neuromorphic Spiking Array Processors: A quest to mimic the brain

Authors: Chetan Singh Thakur, Jamal Molin, Gert Cauwenberghs, Giacomo Indiveri, Kundan Kumar, Ning Qiao, Johannes Schemmel, Runchun Wang, Elisabetta Chicca, Jennifer Olson Hasler, Jae-sun Seo, Shimeng Yu, Yu Cao, André van Schaik, Ralph Etienne-Cummings

Abstract: Neuromorphic engineering (NE) encompasses a diverse range of approaches to information processing that are inspired by neurobiological systems, and this feature distinguishes neuromorphic systems from conventional computing systems. The brain has evolved over billions of years to solve difficult engineering problems by using efficient, parallel, low-power computation. The goal of NE is to design s… ▽ More Neuromorphic engineering (NE) encompasses a diverse range of approaches to information processing that are inspired by neurobiological systems, and this feature distinguishes neuromorphic systems from conventional computing systems. The brain has evolved over billions of years to solve difficult engineering problems by using efficient, parallel, low-power computation. The goal of NE is to design systems capable of brain-like computation. Numerous large-scale neuromorphic projects have emerged recently. This interdisciplinary field was listed among the top 10 technology breakthroughs of 2014 by the MIT Technology Review and among the top 10 emerging technologies of 2015 by the World Economic Forum. NE has two-way goals: one, a scientific goal to understand the computational properties of biological neural systems by using models implemented in integrated circuits (ICs); second, an engineering goal to exploit the known properties of biological systems to design and implement efficient devices for engineering applications. Building hardware neural emulators can be extremely useful for simulating large-scale neural models to explain how intelligent behavior arises in the brain. The principle advantages of neuromorphic emulators are that they are highly energy efficient, parallel and distributed, and require a small silicon area. Thus, compared to conventional CPUs, these neuromorphic emulators are beneficial in many engineering applications such as for the porting of deep learning algorithms for various recognitions tasks. In this review article, we describe some of the most significant neuromorphic spiking emulators, compare the different architectures and approaches used by them, illustrate their advantages and drawbacks, and highlight the capabilities that each can deliver to neural modelers. △ Less

Submitted 22 May, 2018; originally announced May 2018.

An Ultralow Leakage Synaptic Scaling Homeostatic Plasticity Circuit With Configurable Time Scales up to 100 ks

Authors: Ning Qiao, Chiara Bartolozzi, Giacomo Indiveri

Abstract: Homeostatic plasticity is a stabilizing mechanism commonly observed in real neural systems that allows neurons to maintain their activity around a functional operating point. This phenomenon can be used in neuromorphic systems to compensate for slowly changing conditions or chronic shifts in the system configuration. However, to avoid interference with other adaptation or learning processes active… ▽ More Homeostatic plasticity is a stabilizing mechanism commonly observed in real neural systems that allows neurons to maintain their activity around a functional operating point. This phenomenon can be used in neuromorphic systems to compensate for slowly changing conditions or chronic shifts in the system configuration. However, to avoid interference with other adaptation or learning processes active in the neuromorphic system, it is important that the homeostatic plasticity mechanism operates on time scales that are much longer than conventional synaptic plasticity ones. In this paper we present an ultra-low leakage circuit, integrated into an automatic gain control scheme, that can implement the synaptic scaling homeostatic process over extremely long time scales. Synaptic scaling consists in globally scaling the synaptic weights of all synapses impinging onto a neuron maintaining their relative differences, to preserve the effects of learning. The scheme we propose controls the global gain of analog log-domain synapse circuits to keep the neuron's average firing rate constant around a set operating point, over extremely long time scales. To validate the proposed scheme, we implemented the ultra-low leakage synaptic scaling homeostatic plasticity circuit in a standard 0.18 $μ$m Complementary Metal-Oxide Semiconductor (CMOS) process, and integrated it in an array of dynamic synapses connected to an adaptive integrate and fire neuron. The circuit occupies a silicon area of 84 $μ$m x 22 $μ$m and consumes approximately 10.8 nW with a 1.8 V supply voltage. We present experimental results from the homeostatic circuit and demonstrate how it can be configured to exhibit time scales of up to 100 kilo-seconds, thanks to a controllable leakage current that can be scaled down to 0.45 atto-Amperes (2.8 electrons/s). △ Less

Submitted 15 November, 2017; v1 submitted 17 September, 2017; originally announced September 2017.

MSC Class: 68T05; 82C32; 93E35 ACM Class: B.7.1; C.1.3; C.1.4; F.1.1

A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)

Authors: Saber Moradi, Ning Qiao, Fabio Stefanini, Giacomo Indiveri

Abstract: Neuromorphic computing systems comprise networks of neurons that use asynchronous events for both computation and communication. This type of representation offers several advantages in terms of bandwidth and power consumption in neuromorphic electronic systems. However, managing the traffic of asynchronous events in large scale systems is a daunting task, both in terms of circuit complexity and m… ▽ More Neuromorphic computing systems comprise networks of neurons that use asynchronous events for both computation and communication. This type of representation offers several advantages in terms of bandwidth and power consumption in neuromorphic electronic systems. However, managing the traffic of asynchronous events in large scale systems is a daunting task, both in terms of circuit complexity and memory requirements. Here we present a novel routing methodology that employs both hierarchical and mesh routing strategies and combines heterogeneous memory structures for minimizing both memory requirements and latency, while maximizing programming flexibility to support a wide range of event-based neural network architectures, through parameter configuration. We validated the proposed scheme in a prototype multi-core neuromorphic processor chip that employs hybrid analog/digital circuits for emulating synapse and neuron dynamics together with asynchronous digital circuits for managing the address-event traffic. We present a theoretical analysis of the proposed connectivity scheme, describe the methods and circuits used to implement such scheme, and characterize the prototype chip. Finally, we demonstrate the use of the neuromorphic processor with a convolutional neural network for the real-time classification of visual symbols being flashed to a dynamic vision sensor (DVS) at high speed. △ Less

Submitted 16 August, 2017; v1 submitted 14 August, 2017; originally announced August 2017.

Comments: 17 pages, 14 figures