-
Symmetric silicon microring resonator optical crossbar array for accelerated inference and training in deep learning
Authors:
Rui Tang,
Shuhei Ohno,
Ken Tanizawa,
Kazuhiro Ikeda,
Makoto Okano,
Kasidit Toprasertpong,
Shinichi Takagi,
Mitsuru Takenaka
Abstract:
Photonic integrated circuits are emerging as a promising platform for accelerating matrix multiplications in deep learning, leveraging the inherent parallel nature of light. Although various schemes have been proposed and demonstrated to realize such photonic matrix accelerators, the in-situ training of artificial neural networks using photonic accelerators remains challenging due to the difficult…
▽ More
Photonic integrated circuits are emerging as a promising platform for accelerating matrix multiplications in deep learning, leveraging the inherent parallel nature of light. Although various schemes have been proposed and demonstrated to realize such photonic matrix accelerators, the in-situ training of artificial neural networks using photonic accelerators remains challenging due to the difficulty of direct on-chip backpropagation on a photonic chip. In this work, we propose a silicon microring resonator (MRR) optical crossbar array with a symmetric structure that allows for simple on-chip backpropagation, potentially enabling the acceleration of both the inference and training phases of deep learning. We demonstrate a $4 \times 4$ circuit on a Si-on-insulator (SOI) platform and use it to perform inference tasks of a simple neural network for classifying Iris flowers, achieving a classification accuracy of 93.3%. Subsequently, we train the neural network using simulated on-chip backpropagation and achieve an accuracy of 91.1% in the same inference task after training. Furthermore, we simulate a convolutional neural network (CNN) for handwritten digit recognition, using a $9 \times 9$ MRR crossbar array to perform the convolution operations. This work contributes to the realization of compact and energy-efficient photonic accelerators for deep learning.
△ Less
Submitted 1 June, 2024; v1 submitted 29 January, 2024;
originally announced January 2024.
-
Disaster Anomaly Detector via Deeper FCDDs for Explainable Initial Responses
Authors:
Takato Yasuno,
Masahiro Okano,
Junichiro Fujii
Abstract:
Extreme natural disasters can have devastating effects on both urban and rural areas. In any disaster event, an initial response is the key to rescue within 72 hours and prompt recovery. During the initial stage of disaster response, it is important to quickly assess the damage over a wide area and identify priority areas. Among machine learning algorithms, deep anomaly detection is effective in d…
▽ More
Extreme natural disasters can have devastating effects on both urban and rural areas. In any disaster event, an initial response is the key to rescue within 72 hours and prompt recovery. During the initial stage of disaster response, it is important to quickly assess the damage over a wide area and identify priority areas. Among machine learning algorithms, deep anomaly detection is effective in detecting devastation features that are different from everyday features. In addition, explainable computer vision applications should justify the initial responses. In this paper, we propose an anomaly detection application utilizing deeper fully convolutional data descriptions (FCDDs), that enables the localization of devastation features and visualization of damage-marked heatmaps. More specifically, we show numerous training and test results for a dataset AIDER with the four disaster categories: collapsed buildings, traffic incidents, fires, and flooded areas. We also implement ablation studies of anomalous class imbalance and the data scale competing against the normal class. Our experiments provide results of high accuracies over 95% for F1. Furthermore, we found that the deeper FCDD with a VGG16 backbone consistently outperformed other baselines CNN27, ResNet101, and Inceptionv3. This study presents a new solution that offers a disaster anomaly detection application for initial responses with higher accuracy and devastation explainability, providing a novel contribution to the prompt disaster recovery problem in the research area of anomaly scene understanding. Finally, we discuss future works to improve more robust, explainable applications for effective initial responses.
△ Less
Submitted 12 June, 2023; v1 submitted 4 June, 2023;
originally announced June 2023.
-
Wooden Sleeper Deterioration Detection for Rural Railway Prognostics Using Unsupervised Deeper FCDDs
Authors:
Takato Yasuno,
Masahiro Okano,
Junichiro Fujii
Abstract:
Maintaining high standards for user safety during daily railway operations is crucial for railway managers. To aid in this endeavor, top- or side-view cameras and GPS positioning systems have facilitated progress toward automating periodic inspections of defective features and assessing the deteriorating status of railway components. However, collecting data on deteriorated status can be time-cons…
▽ More
Maintaining high standards for user safety during daily railway operations is crucial for railway managers. To aid in this endeavor, top- or side-view cameras and GPS positioning systems have facilitated progress toward automating periodic inspections of defective features and assessing the deteriorating status of railway components. However, collecting data on deteriorated status can be time-consuming and requires repeated data acquisition because of the extreme temporal occurrence imbalance. In supervised learning, thousands of paired data sets containing defective raw images and annotated labels are required. However, the one-class classification approach offers the advantage of requiring fewer images to optimize parameters for training normal and anomalous features. The deeper fully-convolutional data descriptions (FCDDs) were applicable to several damage data sets of concrete/steel components in structures, and fallen tree, and wooden building collapse in disasters. However, it is not yet known to feasible to railway components. In this study, we devised a prognostic discriminator pipeline to automate one-class damage classification using the deeper FCDDs for defective railway components. We also performed ablation studies of the deeper backbone based on convolutional neural networks (CNNs). Furthermore, we visualized deterioration features by using transposed Gaussian upsampling. We demonstrated our application to railway inspection using a video acquisition dataset of railway track from backward view at a cloudy and sunny scene. Finally, we examined the usability of our approach for prognostics and future work on railway inspection.
△ Less
Submitted 27 May, 2023; v1 submitted 8 May, 2023;
originally announced May 2023.
-
One-class Damage Detector Using Deeper Fully-Convolutional Data Descriptions for Civil Application
Authors:
Takato Yasuno,
Masahiro Okano,
Junichiro Fujii
Abstract:
Infrastructure managers must maintain high standards to ensure user satisfaction during the lifecycle of infrastructures. Surveillance cameras and visual inspections have enabled progress in automating the detection of anomalous features and assessing the occurrence of deterioration. However, collecting damage data is typically time consuming and requires repeated inspections. The one-class damage…
▽ More
Infrastructure managers must maintain high standards to ensure user satisfaction during the lifecycle of infrastructures. Surveillance cameras and visual inspections have enabled progress in automating the detection of anomalous features and assessing the occurrence of deterioration. However, collecting damage data is typically time consuming and requires repeated inspections. The one-class damage detection approach has an advantage in that normal images can be used to optimize model parameters. Additionally, visual evaluation of heatmaps enables us to understand localized anomalous features. The authors highlight damage vision applications utilized in the robust property and localized damage explainability. First, we propose a civil-purpose application for automating one-class damage detection reproducing a fully convolutional data description (FCDD) as a baseline model. We have obtained accurate and explainable results demonstrating experimental studies on concrete damage and steel corrosion in civil engineering. Additionally, to develop a more robust application, we applied our method to another outdoor domain that contains complex and noisy backgrounds using natural disaster datasets collected using various devices. Furthermore, we propose a valuable solution of deeper FCDDs focusing on other powerful backbones to improve the performance of damage detection and implement ablation studies on disaster datasets. The key results indicate that the deeper FCDDs outperformed the baseline FCDD on datasets representing natural disaster damage caused by hurricanes, typhoons, earthquakes, and four-event disasters.
△ Less
Submitted 8 May, 2023; v1 submitted 3 March, 2023;
originally announced March 2023.
-
MN-Pair Contrastive Damage Representation and Clustering for Prognostic Explanation
Authors:
Takato Yasuno,
Masahiro Okano,
Junichiro Fujii
Abstract:
For infrastructure inspections, damage representation does not constantly match the predefined classes of damage grade, resulting in detailed clusters of unseen damages or more complex clusters from overlapped space between two grades. The damage representation has fundamentally complex features; consequently, not all the damage classes can be perfectly predefined. The proposed MN-pair contrastive…
▽ More
For infrastructure inspections, damage representation does not constantly match the predefined classes of damage grade, resulting in detailed clusters of unseen damages or more complex clusters from overlapped space between two grades. The damage representation has fundamentally complex features; consequently, not all the damage classes can be perfectly predefined. The proposed MN-pair contrastive learning method helps to explore an embedding damage representation beyond the predefined classes by including more detailed clusters. It maximizes both the similarity of M-1 positive images close to an anchor and dissimilarity of N-1 negative images using both weighting loss functions. It learns faster than the N-pair algorithm using one positive image. We proposed a pipeline to obtain the damage representation and used a density-based clustering on a 2-D reduction space to automate finer cluster discrimination. We also visualized the explanation of the damage feature using Grad-CAM for MN-pair damage metric learning. We demonstrated our method in three experimental studies: steel product defect, concrete crack, and the effectiveness of our method and discuss future works.
△ Less
Submitted 26 March, 2023; v1 submitted 15 January, 2023;
originally announced January 2023.
-
VAE-iForest: Auto-encoding Reconstruction and Isolation-based Anomalies Detecting Fallen Objects on Road Surface
Authors:
Takato Yasuno,
Junichiro Fujii,
Riku Ogata,
Masahiro Okano
Abstract:
In road monitoring, it is an important issue to detect changes in the road surface at an early stage to prevent damage to third parties. The target of the falling object may be a fallen tree due to the external force of a flood or an earthquake, and falling rocks from a slope. Generative deep learning is possible to flexibly detect anomalies of the falling objects on the road surface. We prototype…
▽ More
In road monitoring, it is an important issue to detect changes in the road surface at an early stage to prevent damage to third parties. The target of the falling object may be a fallen tree due to the external force of a flood or an earthquake, and falling rocks from a slope. Generative deep learning is possible to flexibly detect anomalies of the falling objects on the road surface. We prototype a method that combines auto-encoding reconstruction and isolation-based anomaly detector in application for road surface monitoring. Actually, we apply our method to a set of test images that fallen objects is located on the raw inputs added with fallen stone and plywood, and that snow is covered on the winter road. Finally we mention the future works for practical purpose application.
△ Less
Submitted 2 March, 2022;
originally announced March 2022.
-
Flood Inflow Forecast Using L2-norm Ensemble Weighting Sea Surface Feature
Authors:
Takato Yasuno,
Masazumi Amakata,
Junichiro Fujii,
Masahiro Okano,
Riku Ogata
Abstract:
It is important to forecast dam inflow for flood damage mitigation. The hydrograph provides critical information such as the start time, peak level, and volume. Particularly, dam management requires a 6-h lead time of the dam inflow forecast based on a future hydrograph. The authors propose novel target inflow weights to create an ocean feature vector extracted from the analyzed images of the sea…
▽ More
It is important to forecast dam inflow for flood damage mitigation. The hydrograph provides critical information such as the start time, peak level, and volume. Particularly, dam management requires a 6-h lead time of the dam inflow forecast based on a future hydrograph. The authors propose novel target inflow weights to create an ocean feature vector extracted from the analyzed images of the sea surface. We extracted 4,096 elements of the dimension vector in the fc6 layer of the pre-trained VGG16 network. Subsequently, we reduced it to three dimensions of t-SNE. Furthermore, we created the principal component of the sea temperature weights using PCA. We found that these weights contribute to the stability of predictor importance by numerical experiments. As base regression models, we calibrate the least squares with kernel expansion, the quantile random forest minimized out-of bag error, and the support vector regression with a polynomial kernel. When we compute the predictor importance, we visualize the stability of each variable importance introduced by our proposed weights, compared with other results without weights. We apply our method to a dam at Kanto region in Japan and focus on the trained term from 2007 to 2018, with a limited flood term from June to October. We test the accuracy over the 2019 flood term. Finally, we present the applied results and further statistical learning for unknown flood forecast.
△ Less
Submitted 20 July, 2022; v1 submitted 6 December, 2021;
originally announced December 2021.
-
Natural Disaster Classification using Aerial Photography Explainable for Typhoon Damaged Feature
Authors:
Takato Yasuno,
Masazumi Amakata,
Masahiro Okano
Abstract:
Recent years, typhoon damages has become social problem owing to climate change. In 9 September 2019, Typhoon Faxai passed on the Chiba in Japan, whose damages included with electric provision stop because of strong wind recorded on the maximum 45 meter per second. A large amount of tree fell down, and the neighbor electric poles also fell down at the same time. These disaster features have caused…
▽ More
Recent years, typhoon damages has become social problem owing to climate change. In 9 September 2019, Typhoon Faxai passed on the Chiba in Japan, whose damages included with electric provision stop because of strong wind recorded on the maximum 45 meter per second. A large amount of tree fell down, and the neighbor electric poles also fell down at the same time. These disaster features have caused that it took 18 days for recovery longer than past ones. Immediate responses are important for faster recovery. As long as we can, aerial survey for global screening of devastated region would be required for decision support to respond where to recover ahead. This paper proposes a practical method to visualize the damaged areas focused on the typhoon disaster features using aerial photography. This method can classify eight classes which contains land covers without damages and areas with disaster. Using target feature class probabilities, we can visualize disaster feature map to scale a color range. Furthermore, we can realize explainable map on each unit grid images to compute the convolutional activation map using Grad-CAM. We demonstrate case studies applied to aerial photographs recorded at the Chiba region after typhoon.
△ Less
Submitted 16 November, 2020; v1 submitted 21 April, 2020;
originally announced April 2020.
