-
Measurement-driven neural-network training for integrated magnetic tunnel junction arrays
Authors:
William A. Borders,
Advait Madhavan,
Matthew W. Daniels,
Vasileia Georgiou,
Martin Lueker-Boden,
Tiffany S. Santos,
Patrick M. Braganca,
Mark D. Stiles,
Jabez J. McClelland,
Brian D. Hoskins
Abstract:
The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. An inevitability of transferring neural networks onto hardware is that non-idealities such as…
▽ More
The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. An inevitability of transferring neural networks onto hardware is that non-idealities such as device-to-device variations or poor device yield impact performance. Methods such as hardware-aware training, where substrate non-idealities are incorporated during network training, are one way to recover performance at the cost of solution generality. In this work, we demonstrate inference on hardware neural networks consisting of 20,000 magnetic tunnel junction arrays integrated on a complementary metal-oxide-semiconductor chips that closely resembles market-ready spin transfer-torque magnetoresistive random access memory technology. Using 36 dies, each containing a crossbar array with its own non-idealities, we show that even a small number of defects in physically mapped networks significantly degrades the performance of networks trained without defects and show that, at the cost of generality, hardware-aware training accounting for specific defects on each die can recover to comparable performance with ideal networks. We then demonstrate a robust training method that extends hardware-aware training to statistics-aware training, producing network weights that perform well on most defective dies regardless of their specific defect locations. When evaluated on the 36 physical dies, statistics-aware trained solutions can achieve a mean misclassification error on the MNIST dataset that differs from the software-baseline by only 2 %. This statistics-aware training method could be generalized to networks with many layers that are mapped to hardware suited for industry-ready applications.
△ Less
Submitted 14 May, 2024; v1 submitted 11 December, 2023;
originally announced December 2023.
-
MoRSE: Deep Learning-based Arm Gesture Recognition for Search and Rescue Operations
Authors:
Panagiotis Kasnesis,
Christos Chatzigeorgiou,
Dimitrios G. Kogias,
Charalampos Z. Patrikakis,
Harris V. Georgiou,
Aspasia Tzeletopoulou
Abstract:
Efficient and quick remote communication in search and rescue operations can be life-saving for the first responders. However, while operating on the field means of communication based on text, image and audio are not suitable for several disaster scenarios. In this paper, we present a smartwatch-based application, which utilizes a Deep Learning (DL) model, to recognize a set of predefined arm ges…
▽ More
Efficient and quick remote communication in search and rescue operations can be life-saving for the first responders. However, while operating on the field means of communication based on text, image and audio are not suitable for several disaster scenarios. In this paper, we present a smartwatch-based application, which utilizes a Deep Learning (DL) model, to recognize a set of predefined arm gestures, maps them into Morse code via vibrations enabling remote communication amongst first responders. The model performance was evaluated by training it using 4,200 gestures performed by 7 subjects (cross-validation) wearing a smartwatch on their dominant arm. Our DL model relies on convolutional pooling and surpasses the performance of existing DL approaches and common machine learning classifiers, obtaining gesture recognition accuracy above 95%. We conclude by discussing the results and providing future directions.
△ Less
Submitted 15 October, 2022;
originally announced October 2022.
-
On the optimality of ternary arithmetic for compactness and hardware design
Authors:
Harris V. Georgiou
Abstract:
In this paper, the optimality of ternary arithmetic is investigated under strict mathematical formulation. The arithmetic systems are presented in generic form, as the means to encode numeric values, and the choice of radix is asserted as the main parameter to assess the efficiency of the representation, in terms of information compactness and estimated implementation cost in hardware. Using prope…
▽ More
In this paper, the optimality of ternary arithmetic is investigated under strict mathematical formulation. The arithmetic systems are presented in generic form, as the means to encode numeric values, and the choice of radix is asserted as the main parameter to assess the efficiency of the representation, in terms of information compactness and estimated implementation cost in hardware. Using proper formulations for the optimization task, the universal constant 'e' (base of natural logarithms) is proven as the most efficient radix and ternary is asserted as the closest integer choice.
△ Less
Submitted 25 October, 2016;
originally announced November 2016.
-
Identification of refugee influx patterns in Greece via model-theoretic analysis of daily arrivals
Authors:
Harris V. Georgiou
Abstract:
The refugee crisis is perhaps the single most challenging problem for Europe today. Hundreds of thousands of people have already traveled across dangerous sea passages from Turkish shores to Greek islands, resulting in thousands of dead and missing, despite the best rescue efforts from both sides. One of the main reasons is the total lack of any early warning-alerting system, which could provide s…
▽ More
The refugee crisis is perhaps the single most challenging problem for Europe today. Hundreds of thousands of people have already traveled across dangerous sea passages from Turkish shores to Greek islands, resulting in thousands of dead and missing, despite the best rescue efforts from both sides. One of the main reasons is the total lack of any early warning-alerting system, which could provide some preparation time for the prompt and effective deployment of resources at the hot zones. This work is such an attempt for a systemic analysis of the refugee influx in Greece, aiming at (a) the statistical and signal-level characterization of the smuggling networks and (b) the formulation and preliminary assessment of such models for predictive purposes, i.e., as the basis of such an early warning-alerting protocol. To our knowledge, this is the first-ever attempt to design such a system, since this refugee crisis itself and its geographical properties are unique (intense event handling, little or no warning). The analysis employs a wide range of statistical, signal-based and matrix factorization (decomposition) techniques, including linear & linear-cosine regression, spectral analysis, ARMA, SVD, Probabilistic PCA, ICA, K-SVD for Dictionary Learning, as well as fractal dimension analysis. It is established that the behavioral patterns of the smuggling networks closely match (as expected) the regular burst and pause periods of store-and-forward networks in digital communications. There are also major periodic trends in the range of 6.2-6.5 days and strong correlations in lags of four or more days, with distinct preference in the Sunday-Monday 48-hour time frame. These results show that such models can be used successfully for short-term forecasting of the influx intensity, producing an invaluable operational asset for planners, decision-makers and first-responders.
△ Less
Submitted 9 May, 2016;
originally announced May 2016.
-
Elements of Game Theory - Part I: Foundations, acts and mechanisms
Authors:
Harris V. Georgiou
Abstract:
In this paper, a gentle introduction to Game Theory is presented in the form of basic concepts and examples. Minimax and Nash's theorem are introduced as the formal definitions for optimal strategies and equilibria in zero-sum and nonzero-sum games. Several elements of cooperative gaming, coalitions, voting ensembles, voting power and collective efficiency are described in brief. Analytical (matri…
▽ More
In this paper, a gentle introduction to Game Theory is presented in the form of basic concepts and examples. Minimax and Nash's theorem are introduced as the formal definitions for optimal strategies and equilibria in zero-sum and nonzero-sum games. Several elements of cooperative gaming, coalitions, voting ensembles, voting power and collective efficiency are described in brief. Analytical (matrix) and extended (tree-graph) forms of game representation is illustrated as the basic tools for identifying optimal strategies and "solutions" in games of any kind. Next, a typology of four standard nonzero-sum games is investigated, analyzing the Nash equilibria and the optimal strategies in each case. Signaling, stance and third-party intermediates are described as very important properties when analyzing strategic moves, while credibility and reputation is described as crucial factors when signaling promises or threats. Utility is introduced as a generalization of typical cost/gain functions and it is used to explain the incentives of irrational players under the scope of "rational irrationality". Finally, a brief reference is presented for several other more advanced concepts of gaming, including emergence of cooperation, evolutionary stable strategies, two-level games, metagames, hypergames and the Harsanyi transformation.
△ Less
Submitted 16 June, 2015;
originally announced June 2015.
-
Collective decision efficiency and optimal voting mechanisms: A comprehensive overview for multi-classifier models
Authors:
Harris V. Georgiou
Abstract:
A new game-theoretic approach for combining multiple classifiers is proposed. A short introduction in Game Theory and coalitions illustrate the way any collective decision scheme can be viewed as a competitive game of coalitions that are formed naturally when players state their preferences. The winning conditions and the voting power of each player are studied under the scope of voting power indi…
▽ More
A new game-theoretic approach for combining multiple classifiers is proposed. A short introduction in Game Theory and coalitions illustrate the way any collective decision scheme can be viewed as a competitive game of coalitions that are formed naturally when players state their preferences. The winning conditions and the voting power of each player are studied under the scope of voting power indices, as well and the collective competence of the group. Coalitions and power indices are presented in relation to the Condorcet criterion of optimality in voting systems, and weighted Borda count models are asserted as a way to implement them in practice. A special case of coalition games, the weighted majority games (WMG) are presented as a restricted realization in dichotomy choice situations. As a result, the weighted majority rules (WMR), an extended version of the simple majority rules, are asserted as the theoretically optimal and complete solution to this type of coalition gaming. Subsequently, a generalized version of WMRs is suggested as the means to design a voting system that is optimal in the sense of both the correct classification criterion and the Condorcet efficiency criterion. In the scope of Pattern Recognition, a generalized risk-based approach is proposed as the framework upon which any classifier combination scheme can be applied. A new fully adaptive version of WMRs is proposed as a statistically invariant way of adjusting the design process of the optimal WMR to the arbitrary non-symmetrical properties of the underlying feature space. SVM theory is associated with properties and conclusions that emerge from the game-theoretic approach of the classification in general, while the theoretical and practical implications of employing SVM experts in WMR combination schemes are briefly discussed. Finally, a summary of the most important issues for further research is presented.
△ Less
Submitted 7 February, 2015;
originally announced February 2015.
-
Estimating the intrinsic dimension in fMRI space via dataset fractal analysis - Counting the `cpu cores' of the human brain
Authors:
Harris V. Georgiou
Abstract:
Functional Magnetic Resonance Imaging (fMRI) is a powerful non-invasive tool for localizing and analyzing brain activity. This study focuses on one very important aspect of the functional properties of human brain, specifically the estimation of the level of parallelism when performing complex cognitive tasks. Using fMRI as the main modality, the human brain activity is investigated through a pure…
▽ More
Functional Magnetic Resonance Imaging (fMRI) is a powerful non-invasive tool for localizing and analyzing brain activity. This study focuses on one very important aspect of the functional properties of human brain, specifically the estimation of the level of parallelism when performing complex cognitive tasks. Using fMRI as the main modality, the human brain activity is investigated through a purely data-driven signal processing and dimensionality analysis approach. Specifically, the fMRI signal is treated as a multi-dimensional data space and its intrinsic `complexity' is studied via dataset fractal analysis and blind-source separation (BSS) methods. One simulated and two real fMRI datasets are used in combination with Independent Component Analysis (ICA) and fractal analysis for estimating the intrinsic (true) dimensionality, in order to provide data-driven experimental evidence on the number of independent brain processes that run in parallel when visual or visuo-motor tasks are performed. Although this number is can not be defined as a strict threshold but rather as a continuous range, when a specific activation level is defined, a corresponding number of parallel processes or the casual equivalent of `cpu cores' can be detected in normal human brain activity.
△ Less
Submitted 26 October, 2014;
originally announced October 2014.
-
Adaptive detection and severity level characterization algorithm for Obstructive Sleep Apnea Hypopnea Syndrome (OSAHS) via oximetry signal analysis
Authors:
Harris V. Georgiou
Abstract:
In this paper, an abstract definition and formal specification is presented for the task of adaptive-threshold OSAHS events detection and severity characterization. Specifically, a low-level pseudocode is designed for the algorithm of raw oximetry signal pre-processing, calculation of the 'drop' and 'rise' frames in the related time series, detection of valid apnea/hypopnea events via SpO2 saturat…
▽ More
In this paper, an abstract definition and formal specification is presented for the task of adaptive-threshold OSAHS events detection and severity characterization. Specifically, a low-level pseudocode is designed for the algorithm of raw oximetry signal pre-processing, calculation of the 'drop' and 'rise' frames in the related time series, detection of valid apnea/hypopnea events via SpO2 saturation level tracking, as well as calculation of corresponding event rates for OSAHS severity characterization. The designed algorithm can be used as the first module in a machine learning application where these data can be used as inputs or encoded into higher-level statistics (features) for pattern classifiers, in the context of computer-aided or fully automated diagnosis of OSAHS and related pathologies.
△ Less
Submitted 28 August, 2013;
originally announced August 2013.
-
A game-theoretic framework for classifier ensembles using weighted majority voting with local accuracy estimates
Authors:
Harris V. Georgiou,
Michael E. Mavroforakis
Abstract:
In this paper, a novel approach for the optimal combination of binary classifiers is proposed. The classifier combination problem is approached from a Game Theory perspective. The proposed framework of adapted weighted majority rules (WMR) is tested against common rank-based, Bayesian and simple majority models, as well as two soft-output averaging rules. Experiments with ensembles of Support Vect…
▽ More
In this paper, a novel approach for the optimal combination of binary classifiers is proposed. The classifier combination problem is approached from a Game Theory perspective. The proposed framework of adapted weighted majority rules (WMR) is tested against common rank-based, Bayesian and simple majority models, as well as two soft-output averaging rules. Experiments with ensembles of Support Vector Machines (SVM), Ordinary Binary Tree Classifiers (OBTC) and weighted k-nearest-neighbor (w/k-NN) models on benchmark datasets indicate that this new adaptive WMR model, employing local accuracy estimators and the analytically computed optimal weights outperform all the other simple combination rules.
△ Less
Submitted 3 February, 2013;
originally announced February 2013.
