Showing 1–26 of 26 results for author: Yamaguchi, T

Evaluation of Large Language Models for Decision Making in Autonomous Driving

Authors: Kotaro Tanahashi, Yuichi Inoue, Yu Yamaguchi, Hidetatsu Yaginuma, Daiki Shiotsuka, Hiroyuki Shimatani, Kohei Iwamasa, Yoshiaki Inoue, Takafumi Yamaguchi, Koki Igari, Tsukasa Horinouchi, Kento Tokuhiro, Yugo Tokuchi, Shunsuke Aoki

Abstract: Various methods have been proposed for utilizing Large Language Models (LLMs) in autonomous driving. One strategy of using LLMs for autonomous driving involves inputting surrounding objects as text prompts to the LLMs, along with their coordinate and velocity information, and then outputting the subsequent movements of the vehicle. When using LLMs for such purposes, capabilities such as spatial re… ▽ More Various methods have been proposed for utilizing Large Language Models (LLMs) in autonomous driving. One strategy of using LLMs for autonomous driving involves inputting surrounding objects as text prompts to the LLMs, along with their coordinate and velocity information, and then outputting the subsequent movements of the vehicle. When using LLMs for such purposes, capabilities such as spatial recognition and planning are essential. In particular, two foundational capabilities are required: (1) spatial-aware decision making, which is the ability to recognize space from coordinate information and make decisions to avoid collisions, and (2) the ability to adhere to traffic rules. However, quantitative research has not been conducted on how accurately different types of LLMs can handle these problems. In this study, we quantitatively evaluated these two abilities of LLMs in the context of autonomous driving. Furthermore, to conduct a Proof of Concept (POC) for the feasibility of implementing these abilities in actual vehicles, we developed a system that uses LLMs to drive a vehicle. △ Less

Submitted 11 December, 2023; originally announced December 2023.

Comments: Accepted at the 2023 Symposium on Machine Learning for Autonomous Driving collocated with NeurIPS

cuQuantum SDK: A High-Performance Library for Accelerating Quantum Science

Authors: Harun Bayraktar, Ali Charara, David Clark, Saul Cohen, Timothy Costa, Yao-Lung L. Fang, Yang Gao, Jack Guan, John Gunnels, Azzam Haidar, Andreas Hehn, Markus Hohnerbach, Matthew Jones, Tom Lubowe, Dmitry Lyakh, Shinya Morino, Paul Springer, Sam Stanwyck, Igor Terentyev, Satya Varadhan, Jonathan Wong, Takuma Yamaguchi

Abstract: We present the NVIDIA cuQuantum SDK, a state-of-the-art library of composable primitives for GPU-accelerated quantum circuit simulations. As the size of quantum devices continues to increase, making their classical simulation progressively more difficult, the availability of fast and scalable quantum circuit simulators becomes vital for quantum algorithm developers, as well as quantum hardware eng… ▽ More We present the NVIDIA cuQuantum SDK, a state-of-the-art library of composable primitives for GPU-accelerated quantum circuit simulations. As the size of quantum devices continues to increase, making their classical simulation progressively more difficult, the availability of fast and scalable quantum circuit simulators becomes vital for quantum algorithm developers, as well as quantum hardware engineers focused on the validation and optimization of quantum devices. The cuQuantum SDK was created to accelerate and scale up quantum circuit simulators developed by the quantum information science community by enabling them to utilize efficient scalable software building blocks optimized for NVIDIA GPU platforms. The functional building blocks provided cover the needs of both state vector- and tensor network- based simulators, including approximate tensor network simulation methods based on matrix product state, projected entangled pair state, and other factorized tensor representations. By leveraging the enormous computing power of the latest NVIDIA GPU architectures, quantum circuit simulators that have adopted the cuQuantum SDK demonstrate significant acceleration, compared to CPU-only execution, for both the state vector and tensor network simulation methods. Furthermore, by utilizing the parallel primitives available in the cuQuantum SDK, one can easily transition to distributed GPU-accelerated platforms, including those furnished by cloud service providers and high-performance computing systems deployed by supercomputing centers, extending the scale of possible quantum circuit simulations. The rich capabilities provided by the SDK are conveniently made available via both Python and C application programming interfaces, where the former is directly targeting a broad Python quantum community and the latter allows tight integration with simulators written in any programming language. △ Less

Submitted 3 August, 2023; originally announced August 2023.

Comments: paper accepted at QCE 2023, journal reference will be updated whenever available

MSC Class: 68Q12; 68Q09; 81P68;

MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Results

Authors: Yuki Kondo, Norimichi Ukita, Takayuki Yamaguchi, Hao-Yu Hou, Mu-Yi Shen, Chia-Chi Hsu, En-Ming Huang, Yu-Chen Huang, Yu-Cheng Xia, Chien-Yao Wang, Chun-Yi Lee, Da Huo, Marc A. Kastner, Tingwei Liu, Yasutomo Kawanishi, Takatsugu Hirayama, Takahiro Komamizu, Ichiro Ide, Yosuke Shinya, Xinyao Liu, Guang Liang, Syusuke Yasui

Abstract: Small Object Detection (SOD) is an important machine vision topic because (i) a variety of real-world applications require object detection for distant objects and (ii) SOD is a challenging task due to the noisy, blurred, and less-informative image appearances of small objects. This paper proposes a new SOD dataset consisting of 39,070 images including 137,121 bird instances, which is called the S… ▽ More Small Object Detection (SOD) is an important machine vision topic because (i) a variety of real-world applications require object detection for distant objects and (ii) SOD is a challenging task due to the noisy, blurred, and less-informative image appearances of small objects. This paper proposes a new SOD dataset consisting of 39,070 images including 137,121 bird instances, which is called the Small Object Detection for Spotting Birds (SOD4SB) dataset. The detail of the challenge with the SOD4SB dataset is introduced in this paper. In total, 223 participants joined this challenge. This paper briefly introduces the award-winning methods. The dataset, the baseline code, and the website for evaluation on the public testset are publicly available. △ Less

Submitted 18 July, 2023; originally announced July 2023.

Comments: This paper is included in the proceedings of the 18th International Conference on Machine Vision Applications (MVA2023). It will be officially published at a later date. Project page : https://www.mva-org.jp/mva2023/challenge

Journal ref: 2023 18th International Conference on Machine Vision and Applications (MVA)

A Neurosymbolic Approach to the Verification of Temporal Logic Properties of Learning enabled Control Systems

Authors: Navid Hashemi, Bardh Hoxha, Tomoya Yamaguchi, Danil Prokhorov, Geogios Fainekos, Jyotirmoy Deshmukh

Abstract: Signal Temporal Logic (STL) has become a popular tool for expressing formal requirements of Cyber-Physical Systems (CPS). The problem of verifying STL properties of neural network-controlled CPS remains a largely unexplored problem. In this paper, we present a model for the verification of Neural Network (NN) controllers for general STL specifications using a custom neural architecture where we ma… ▽ More Signal Temporal Logic (STL) has become a popular tool for expressing formal requirements of Cyber-Physical Systems (CPS). The problem of verifying STL properties of neural network-controlled CPS remains a largely unexplored problem. In this paper, we present a model for the verification of Neural Network (NN) controllers for general STL specifications using a custom neural architecture where we map an STL formula into a feed-forward neural network with ReLU activation. In the case where both our plant model and the controller are ReLU-activated neural networks, we reduce the STL verification problem to reachability in ReLU neural networks. We also propose a new approach for neural network controllers with general activation functions; this approach is a sound and complete verification approach based on computing the Lipschitz constant of the closed-loop control system. We demonstrate the practical efficacy of our techniques on a number of examples of learning-enabled control systems. △ Less

Submitted 6 March, 2023; originally announced March 2023.

Ultrafast single-channel machine vision based on neuro-inspired photonic computing

Authors: Tomoya Yamaguchi, Kohei Arai, Tomoaki Niiyama, Atsushi Uchida, Satoshi Sunada

Abstract: High-speed machine vision is increasing its importance in both scientific and technological applications. Neuro-inspired photonic computing is a promising approach to speed-up machine vision processing with ultralow latency. However, the processing rate is fundamentally limited by the low frame rate of image sensors, typically operating at tens of hertz. Here, we propose an image-sensor-free machi… ▽ More High-speed machine vision is increasing its importance in both scientific and technological applications. Neuro-inspired photonic computing is a promising approach to speed-up machine vision processing with ultralow latency. However, the processing rate is fundamentally limited by the low frame rate of image sensors, typically operating at tens of hertz. Here, we propose an image-sensor-free machine vision framework, which optically processes real-world visual information with only a single input channel, based on a random temporal encoding technique. This approach allows for compressive acquisitions of visual information with a single channel at gigahertz rates, outperforming conventional approaches, and enables its direct photonic processing using a photonic reservoir computer in a time domain. We experimentally demonstrate that the proposed approach is capable of high-speed image recognition and anomaly detection, and furthermore, it can be used for high-speed imaging. The proposed approach is multipurpose and can be extended for a wide range of applications, including tracking, controlling, and capturing sub-nanosecond phenomena. △ Less

Submitted 15 February, 2023; originally announced February 2023.

Comments: 30 pages, 12 figures

Journal ref: Commun Phys 6, 250 (2023)

Timed Partial Order Inference Algorithm

Authors: Kandai Watanabe, Bardh Hoxha, Danil Prokhorov, Georgios Fainekos, Morteza Lahijanian, Sriram Sankaranarayana, Tomoya Yamaguchi

Abstract: In this work, we propose the model of timed partial orders (TPOs) for specifying workflow schedules, especially for modeling manufacturing processes. TPOs integrate partial orders over events in a workflow, specifying ``happens-before'' relations, with timing constraints specified using guards and resets on clocks -- an idea borrowed from timed-automata specifications. TPOs naturally allow us to c… ▽ More In this work, we propose the model of timed partial orders (TPOs) for specifying workflow schedules, especially for modeling manufacturing processes. TPOs integrate partial orders over events in a workflow, specifying ``happens-before'' relations, with timing constraints specified using guards and resets on clocks -- an idea borrowed from timed-automata specifications. TPOs naturally allow us to capture event ordering, along with a restricted but useful class of timing relationships. Next, we consider the problem of mining TPO schedules from workflow logs, which include events along with their time stamps. We demonstrate a relationship between formulating TPOs and the graph-coloring problem, and present an algorithm for learning TPOs with correctness guarantees. We demonstrate our approach on synthetic datasets, including two datasets inspired by real-life applications of aircraft turnaround and gameplay videos of the Overcooked computer game. Our TPO mining algorithm can infer TPOs involving hundreds of events from thousands of data-points within a few seconds. We show that the resulting TPOs provide useful insights into the dependencies and timing constraints for workflows. △ Less

Submitted 5 February, 2023; originally announced February 2023.

Pose Estimation for Human Wearing Loose-Fitting Clothes: Obtaining Ground Truth Posture Using HFR Camera and Blinking LEDs

Authors: Takayoshi Yamaguchi, Dan Mikami, Seiji Matsumura, Naoki Saijo, Makio Kashino

Abstract: Human pose estimation, particularly in athletes, can help improve their performance. However, this estimation is difficult using existing methods, such as human annotation, if the subjects wear loose-fitting clothes such as ski/snowboard wears. This study developed a method for obtaining the ground truth data on two-dimensional (2D) poses of a human wearing loose-fitting clothes. This method uses… ▽ More Human pose estimation, particularly in athletes, can help improve their performance. However, this estimation is difficult using existing methods, such as human annotation, if the subjects wear loose-fitting clothes such as ski/snowboard wears. This study developed a method for obtaining the ground truth data on two-dimensional (2D) poses of a human wearing loose-fitting clothes. This method uses fast-flushing light-emitting diodes (LEDs). The subjects were required to wear loose-fitting clothes and place the LED on the target joints. The LEDs were observed directly using a camera by selecting thin filmy loose-fitting clothes. The proposed method captures the scene at 240 fps by using a high-frame-rate camera and renders two 30 fps image sequences by extracting LED-on and -off frames. The temporal differences between the two video sequences can be ignored, considering the speed of human motion. The LED-on video was used to manually annotate the joints and thus obtain the ground truth data. Additionally, the LED-off video, equivalent to a standard video at 30 fps, confirmed the accuracy of existing machine learning-based methods and manual annotations. Experiments demonstrated that the proposed method can obtain ground truth data for standard RGB videos. Further, it was revealed that neither manual annotation nor the state-of-the-art pose estimator obtains the correct position of target joints. △ Less

Submitted 8 December, 2022; originally announced December 2022.

Comments: Extended abstract of WACV2023 workshop on Computer Vision 4 Winter Sports

Risk-Awareness in Learning Neural Controllers for Temporal Logic Objectives

Authors: Navid Hashemi, Xin Qin, Jyotirmoy V. Deshmukh, Georgios Fainekos, Bardh Hoxha, Danil Prokhorov, Tomoya Yamaguchi

Abstract: In this paper, we consider the problem of synthesizing a controller in the presence of uncertainty such that the resulting closed-loop system satisfies certain hard constraints while optimizing certain (soft) performance objectives. We assume that the hard constraints encoding safety or mission-critical task objectives are expressed using Signal Temporal Logic (STL), while performance is quantifie… ▽ More In this paper, we consider the problem of synthesizing a controller in the presence of uncertainty such that the resulting closed-loop system satisfies certain hard constraints while optimizing certain (soft) performance objectives. We assume that the hard constraints encoding safety or mission-critical task objectives are expressed using Signal Temporal Logic (STL), while performance is quantified using standard cost functions on system trajectories. In order to prioritize the satisfaction of the hard STL constraints, we utilize the framework of control barrier functions (CBFs) and algorithmically obtain CBFs for STL objectives. We assume that the controllers are modeled using neural networks (NNs) and provide an optimization algorithm to learn the optimal parameters for the NN controller that optimize the performance at a user-specified robustness margin for the safety specifications. We use the formalism of risk measures to evaluate the risk incurred by the trade-off between robustness margin of the system and its performance. We demonstrate the efficacy of our approach on well-known difficult examples for nonlinear control such as a quad-rotor and a unicycle, where the mission objectives for each system include hard timing constraints and safety objectives. △ Less

Submitted 13 October, 2022; originally announced October 2022.

Risk-Bounded Control with Kalman Filtering and Stochastic Barrier Functions

Authors: Shakiba Yaghoubi, Georgios Fainekos, Tomoya Yamaguchi, Danil Prokhorov, Bardh Hoxha

Abstract: In this paper, we study Stochastic Control Barrier Functions (SCBFs) to enable the design of probabilistic safe real-time controllers in presence of uncertainties and based on noisy measurements. Our goal is to design controllers that bound the probability of a system failure in finite-time to a given desired value. To that end, we first estimate the system states from the noisy measurements using… ▽ More In this paper, we study Stochastic Control Barrier Functions (SCBFs) to enable the design of probabilistic safe real-time controllers in presence of uncertainties and based on noisy measurements. Our goal is to design controllers that bound the probability of a system failure in finite-time to a given desired value. To that end, we first estimate the system states from the noisy measurements using an Extended Kalman filter, and compute confidence intervals on the filtering errors. Then, we account for filtering errors and derive sufficient conditions on the control input based on the estimated states to bound the probability that the real states of the system enter an unsafe region within a finite time interval. We show that these sufficient conditions are linear constraints on the control input, and, hence, they can be used in tractable optimization problems to achieve safety, in addition to other properties like reachability, and stability. Our approach is evaluated using a simulation of a lane-changing scenario on a highway with dense traffic. △ Less

Submitted 29 December, 2021; originally announced December 2021.

Comments: CDC 2021, 60th Conference on Decision and Control, 7 pages

PerceMon: Online Monitoring for Perception Systems

Authors: Anand Balakrishnan, Jyotirmoy Deshmukh, Bardh Hoxha, Tomoya Yamaguchi, Georgios Fainekos

Abstract: Perception algorithms in autonomous vehicles are vital for the vehicle to understand the semantics of its surroundings, including detection and tracking of objects in the environment. The outputs of these algorithms are in turn used for decision-making in safety-critical scenarios like collision avoidance, and automated emergency braking. Thus, it is crucial to monitor such perception systems at r… ▽ More Perception algorithms in autonomous vehicles are vital for the vehicle to understand the semantics of its surroundings, including detection and tracking of objects in the environment. The outputs of these algorithms are in turn used for decision-making in safety-critical scenarios like collision avoidance, and automated emergency braking. Thus, it is crucial to monitor such perception systems at runtime. However, due to the high-level, complex representations of the outputs of perception systems, it is a challenge to test and verify these systems, especially at runtime. In this paper, we present a runtime monitoring tool, PerceMon that can monitor arbitrary specifications in Timed Quality Temporal Logic (TQTL) and its extensions with spatial operators. We integrate the tool with the CARLA autonomous vehicle simulation environment and the ROS middleware platform while monitoring properties on state-of-the-art object detection and tracking algorithms. △ Less

Submitted 17 August, 2021; originally announced August 2021.

Neural Network Repair with Reachability Analysis

Authors: Xiaodong Yang, Tom Yamaguchi, Hoang-Dung Tran, Bardh Hoxha, Taylor T Johnson, Danil Prokhorov

Abstract: Safety is a critical concern for the next generation of autonomy that is likely to rely heavily on deep neural networks for perception and control. Formally verifying the safety and robustness of well-trained DNNs and learning-enabled systems under attacks, model uncertainties, and sensing errors is essential for safe autonomy. This research proposes a framework to repair unsafe DNNs in safety-cri… ▽ More Safety is a critical concern for the next generation of autonomy that is likely to rely heavily on deep neural networks for perception and control. Formally verifying the safety and robustness of well-trained DNNs and learning-enabled systems under attacks, model uncertainties, and sensing errors is essential for safe autonomy. This research proposes a framework to repair unsafe DNNs in safety-critical systems with reachability analysis. The repair process is inspired by adversarial training which has demonstrated high effectiveness in improving the safety and robustness of DNNs. Different from traditional adversarial training approaches where adversarial examples are utilized from random attacks and may not be representative of all unsafe behaviors, our repair process uses reachability analysis to compute the exact unsafe regions and identify sufficiently representative examples to enhance the efficacy and efficiency of the adversarial training. The performance of our framework is evaluated on two types of benchmarks without safe models as references. One is a DNN controller for aircraft collision avoidance with access to training data. The other is a rocket lander where our framework can be seamlessly integrated with the well-known deep deterministic policy gradient (DDPG) reinforcement learning algorithm. The experimental results show that our framework can successfully repair all instances on multiple safety specifications with negligible performance degradation. In addition, to increase the computational and memory efficiency of the reachability analysis algorithm, we propose a depth-first-search algorithm that combines an existing exact analysis method with an over-approximation approach based on a new set representation. Experimental results show that our method achieves a five-fold improvement in runtime and a two-fold improvement in memory usage compared to exact analysis. △ Less

Submitted 9 August, 2021; originally announced August 2021.

Designing a Location Trace Anonymization Contest

Authors: Takao Murakami, Hiromi Arai, Koki Hamada, Takuma Hatano, Makoto Iguchi, Hiroaki Kikuchi, Atsushi Kuromasa, Hiroshi Nakagawa, Yuichi Nakamura, Kenshiro Nishiyama, Ryo Nojima, Hidenobu Oguri, Chiemi Watanabe, Akira Yamada, Takayasu Yamaguchi, Yuji Yamaoka

Abstract: For a better understanding of anonymization methods for location traces, we have designed and held a location trace anonymization contest that deals with a long trace (400 events per user) and fine-grained locations (1024 regions). In our contest, each team anonymizes her original traces, and then the other teams perform privacy attacks against the anonymized traces. In other words, both defense a… ▽ More For a better understanding of anonymization methods for location traces, we have designed and held a location trace anonymization contest that deals with a long trace (400 events per user) and fine-grained locations (1024 regions). In our contest, each team anonymizes her original traces, and then the other teams perform privacy attacks against the anonymized traces. In other words, both defense and attack compete together, which is close to what happens in real life. Prior to our contest, we show that re-identification alone is insufficient as a privacy risk and that trace inference should be added as an additional risk. Specifically, we show an example of anonymization that is perfectly secure against re-identification and is not secure against trace inference. Based on this, our contest evaluates both the re-identification risk and trace inference risk and analyzes their relationship. Through our contest, we show several findings in a situation where both defense and attack compete together. In particular, we show that an anonymization method secure against trace inference is also secure against re-identification under the presence of appropriate pseudonymization. We also report defense and attack algorithms that won first place, and analyze the utility of anonymized traces submitted by teams in various applications such as POI recommendation and geo-data analysis. △ Less

Submitted 5 September, 2022; v1 submitted 21 July, 2021; originally announced July 2021.

Comments: Accepted at PoPETs (Proceedings on Privacy Enhancing Technologies) 2023

Reachability Analysis of Convolutional Neural Networks

Authors: Xiaodong Yang, Tomoya Yamaguchi, Hoang-Dung Tran, Bardh Hoxha, Taylor T Johnson, Danil Prokhorov

Abstract: Deep convolutional neural networks have been widely employed as an effective technique to handle complex and practical problems. However, one of the fundamental problems is the lack of formal methods to analyze their behavior. To address this challenge, we propose an approach to compute the exact reachable sets of a network given an input domain, where the reachable set is represented by the face… ▽ More Deep convolutional neural networks have been widely employed as an effective technique to handle complex and practical problems. However, one of the fundamental problems is the lack of formal methods to analyze their behavior. To address this challenge, we propose an approach to compute the exact reachable sets of a network given an input domain, where the reachable set is represented by the face lattice structure. Besides the computation of reachable sets, our approach is also capable of backtracking to the input domain given an output reachable set. Therefore, a full analysis of a network's behavior can be realized. In addition, an approach for fast analysis is also introduced, which conducts fast computation of reachable sets by considering selected sensitive neurons in each layer. The exact pixel-level reachability analysis method is evaluated on a CNN for the CIFAR10 dataset and compared to related works. The fast analysis method is evaluated over a CNN CIFAR10 dataset and VGG16 architecture for the ImageNet dataset. △ Less

Submitted 22 June, 2021; originally announced June 2021.

Robotic Assistant Agent for Student and Machine Co-Learning on AI-FML Practice with AIoT Application

Authors: Chang-Shing Lee, Mei-Hui Wang, Zong-Han Ciou, Rin-Pin Chang, Chun-Hao Tsai, Shen-Chien Chen, Tzong-Xiang Huang, Eri Sato-Shimokawara, Toru Yamaguchi

Abstract: In this paper, the Robotic Assistant Agent for student and machine co-learning on AI-FML practice with AIoT application is presented. The structure of AI-FML contains three parts, including fuzzy logic, neural network, and evolutionary computation. Besides, the Robotic Assistant Agent (RAA) can assist students and machines in co-learning English and AI-FML practice based on the robot Kebbi Air and… ▽ More In this paper, the Robotic Assistant Agent for student and machine co-learning on AI-FML practice with AIoT application is presented. The structure of AI-FML contains three parts, including fuzzy logic, neural network, and evolutionary computation. Besides, the Robotic Assistant Agent (RAA) can assist students and machines in co-learning English and AI-FML practice based on the robot Kebbi Air and AIoT-FML learning tool. Since Sept. 2019, we have introduced an Intelligent Speaking English Assistant (ISEA) App and AI-FML platform to English and computer science learning classes at two elementary schools in Taiwan. We use the collected English-learning data to train a predictive regression model based on students' monthly examination scores. In Jan. 2021, we further combined the developed AI-FML platform with a novel AIoT-FML learning tool to enhance students' interests in learning English and AI-FML with basic hands-on practice. The proposed RAA is responsible for reasoning students' learning performance and showing the results on the AIoT-FML learning tool after communicating with the AI-FML platform. The experimental results and the collection of students' feedback show that this kind of learning model is popular with elementary-school and high-school students, and the learning performance of elementary-school students is improved. △ Less

Submitted 11 May, 2021; originally announced May 2021.

Safe Navigation in Human Occupied Environments Using Sampling and Control Barrier Functions

Authors: Keyvan Majd, Shakiba Yaghoubi, Tomoya Yamaguchi, Bardh Hoxha, Danil Prokhorov, Georgios Fainekos

Abstract: Sampling-based methods such as Rapidly-exploring Random Trees (RRTs) have been widely used for generating motion paths for autonomous mobile systems. In this work, we extend time-based RRTs with Control Barrier Functions (CBFs) to generate, safe motion plans in dynamic environments with many pedestrians. Our framework is based upon a human motion prediction model which is well suited for indoor na… ▽ More Sampling-based methods such as Rapidly-exploring Random Trees (RRTs) have been widely used for generating motion paths for autonomous mobile systems. In this work, we extend time-based RRTs with Control Barrier Functions (CBFs) to generate, safe motion plans in dynamic environments with many pedestrians. Our framework is based upon a human motion prediction model which is well suited for indoor narrow environments. We demonstrate our approach on a high-fidelity model of the Toyota Human Support Robot navigating in narrow corridors. We show in three scenarios that our proposed online method can navigate safely in the presence of moving agents with unknown dynamics. △ Less

Submitted 2 August, 2021; v1 submitted 3 May, 2021; originally announced May 2021.

A Study on AI-FML Robotic Agent for Student Learning Behavior Ontology Construction

Authors: Chang-Shing Lee, Mei-Hui Wang, Wen-Kai Kuan, Zong-Han Ciou, Yi-Lin Tsai, Wei-Shan Chang, Lian-Chao Li, Naoyuki Kubota, Tzong-Xiang Huang, Eri Sato-Shimokawara, Toru Yamaguchi

Abstract: In this paper, we propose an AI-FML robotic agent for student learning behavior ontology construction which can be applied in English speaking and listening domain. The AI-FML robotic agent with the ontology contains the perception intelligence, computational intelligence, and cognition intelligence for analyzing student learning behavior. In addition, there are three intelligent agents, including… ▽ More In this paper, we propose an AI-FML robotic agent for student learning behavior ontology construction which can be applied in English speaking and listening domain. The AI-FML robotic agent with the ontology contains the perception intelligence, computational intelligence, and cognition intelligence for analyzing student learning behavior. In addition, there are three intelligent agents, including a perception agent, a computational agent, and a cognition agent in the AI-FML robotic agent. We deploy the perception agent and the cognition agent on the robot Kebbi Air. Moreover, the computational agent with the Deep Neural Network (DNN) model is performed in the cloud and can communicate with the perception agent and cognition agent via the Internet. The proposed AI-FML robotic agent is applied in Taiwan and tested in Japan. The experimental results show that the agents can be utilized in the human and machine co-learning model for the future education. △ Less

Submitted 27 June, 2020; v1 submitted 17 June, 2020; originally announced June 2020.

Comments: This article has been accepted as a conference paper at CcS 2020 and will be published in IEEE

RTAMT: Online Robustness Monitors from STL

Authors: Dejan Nickovic, Tomoya Yamaguchi

Abstract: We present RTAMT, an online monitoring library for Signal Temporal Logic (STL) and its interface-aware variant (IA-STL), providing both discrete- and dense-time interpretation of the logic. We also introduce RTAMT4ROS, a tool that integrates RTAMT with Robotic Operating System (ROS), a common environment for developing robotic applications. We evaluate RTAMT and RTAMT4ROS on two robotic case studi… ▽ More We present RTAMT, an online monitoring library for Signal Temporal Logic (STL) and its interface-aware variant (IA-STL), providing both discrete- and dense-time interpretation of the logic. We also introduce RTAMT4ROS, a tool that integrates RTAMT with Robotic Operating System (ROS), a common environment for developing robotic applications. We evaluate RTAMT and RTAMT4ROS on two robotic case studies. △ Less

Submitted 24 May, 2020; originally announced May 2020.

Automated eye disease classification method from anterior eye image using anatomical structure focused image classification technique

Authors: Masahiro Oda, Takefumi Yamaguchi, Hideki Fukuoka, Yuta Ueno, Kensaku Mori

Abstract: This paper presents an automated classification method of infective and non-infective diseases from anterior eye images. Treatments for cases of infective and non-infective diseases are different. Distinguishing them from anterior eye images is important to decide a treatment plan. Ophthalmologists distinguish them empirically. Quantitative classification of them based on computer assistance is ne… ▽ More This paper presents an automated classification method of infective and non-infective diseases from anterior eye images. Treatments for cases of infective and non-infective diseases are different. Distinguishing them from anterior eye images is important to decide a treatment plan. Ophthalmologists distinguish them empirically. Quantitative classification of them based on computer assistance is necessary. We propose an automated classification method of anterior eye images into cases of infective or non-infective disease. Anterior eye images have large variations of the eye position and brightness of illumination. This makes the classification difficult. If we focus on the cornea, positions of opacified areas in the corneas are different between cases of the infective and non-infective diseases. Therefore, we solve the anterior eye image classification task by using an object detection approach targeting the cornea. This approach can be said as "anatomical structure focused image classification". We use the YOLOv3 object detection method to detect corneas of infective disease and corneas of non-infective disease. The detection result is used to define a classification result of a image. In our experiments using anterior eye images, 88.3% of images were correctly classified by the proposed method. △ Less

Submitted 4 May, 2020; originally announced May 2020.

Comments: Accepted paper as a poster presentation at SPIE Medical Imaging 2020, Houston, TX, USA

Journal ref: Proceedings of SPIE Medical Imaging 2020: Computer-Aided Diagnosis, Vol.11314, 1131446

Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision

Authors: Denis Gudovskiy, Alec Hodgkinson, Takuya Yamaguchi, Sotaro Tsukizawa

Abstract: Active learning (AL) aims to minimize labeling efforts for data-demanding deep neural networks (DNNs) by selecting the most representative data points for annotation. However, currently used methods are ill-equipped to deal with biased data. The main motivation of this paper is to consider a realistic setting for pool-based semi-supervised AL, where the unlabeled collection of train data is biased… ▽ More Active learning (AL) aims to minimize labeling efforts for data-demanding deep neural networks (DNNs) by selecting the most representative data points for annotation. However, currently used methods are ill-equipped to deal with biased data. The main motivation of this paper is to consider a realistic setting for pool-based semi-supervised AL, where the unlabeled collection of train data is biased. We theoretically derive an optimal acquisition function for AL in this setting. It can be formulated as distribution shift minimization between unlabeled train data and weakly-labeled validation dataset. To implement such acquisition function, we propose a low-complexity method for feature density matching using self-supervised Fisher kernel (FK) as well as several novel pseudo-label estimators. Our FK-based method outperforms state-of-the-art methods on MNIST, SVHN, and ImageNet classification while requiring only 1/10th of processing. The conducted experiments show at least 40% drop in labeling efforts for the biased class-imbalanced data compared to existing methods. △ Less

Submitted 29 February, 2020; originally announced March 2020.

Comments: Accepted to CVPR 2020. Preprint

Smart Home Appliances: Chat with Your Fridge

Authors: Denis Gudovskiy, Gyuri Han, Takuya Yamaguchi, Sotaro Tsukizawa

Abstract: Current home appliances are capable to execute a limited number of voice commands such as turning devices on or off, adjusting music volume or light conditions. Recent progress in machine reasoning gives an opportunity to develop new types of conversational user interfaces for home appliances. In this paper, we apply state-of-the-art visual reasoning model and demonstrate that it is feasible to as… ▽ More Current home appliances are capable to execute a limited number of voice commands such as turning devices on or off, adjusting music volume or light conditions. Recent progress in machine reasoning gives an opportunity to develop new types of conversational user interfaces for home appliances. In this paper, we apply state-of-the-art visual reasoning model and demonstrate that it is feasible to ask a smart fridge about its contents and various properties of the food with close-to-natural conversation experience. Our visual reasoning model answers user questions about existence, count, category and freshness of each product by analyzing photos made by the image sensor inside the smart fridge. Users may chat with their fridge using off-the-shelf phone messenger while being away from home, for example, when shopping in the supermarket. We generate a visually realistic synthetic dataset to train machine learning reasoning model that achieves 95% answer accuracy on test data. We present the results of initial user tests and discuss how we modify distribution of generated questions for model training based on human-in-the-loop guidance. We open source code for the whole system including dataset generation, reasoning model and demonstration scripts. △ Less

Submitted 19 December, 2019; originally announced December 2019.

Comments: NeurIPS 2019 demo track

Einconv: Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networks

Authors: Kohei Hayashi, Taiki Yamaguchi, Yohei Sugawara, Shin-ichi Maeda

Abstract: Tensor decomposition methods are widely used for model compression and fast inference in convolutional neural networks (CNNs). Although many decompositions are conceivable, only CP decomposition and a few others have been applied in practice, and no extensive comparisons have been made between available methods. Previous studies have not determined how many decompositions are available, nor which… ▽ More Tensor decomposition methods are widely used for model compression and fast inference in convolutional neural networks (CNNs). Although many decompositions are conceivable, only CP decomposition and a few others have been applied in practice, and no extensive comparisons have been made between available methods. Previous studies have not determined how many decompositions are available, nor which of them is optimal. In this study, we first characterize a decomposition class specific to CNNs by adopting a flexible graphical notation. The class includes such well-known CNN modules as depthwise separable convolution layers and bottleneck layers, but also previously unknown modules with nonlinear activations. We also experimentally compare the tradeoff between prediction accuracy and time/space complexity for modules found by enumerating all possible decompositions, or by using a neural architecture search. We find some nonlinear decompositions outperform existing ones. △ Less

Submitted 27 November, 2019; v1 submitted 12 August, 2019; originally announced August 2019.

Comments: NeurIPS 2019

A GFML-based Robot Agent for Human and Machine Cooperative Learning on Game of Go

Authors: Chang-Shing Lee, Mei-Hui Wang, Li-Chuang Chen, Yusuke Nojima, Tzong-Xiang Huang, Jinseok Woo, Naoyuki Kubota, Eri Sato-Shimokawara, Toru Yamaguchi

Abstract: This paper applies a genetic algorithm and fuzzy markup language to construct a human and smart machine cooperative learning system on game of Go. The genetic fuzzy markup language (GFML)-based Robot Agent can work on various kinds of robots, including Palro, Pepper, and TMUs robots. We use the parameters of FAIR open source Darkforest and OpenGo AI bots to construct the knowledge base of Open Go… ▽ More This paper applies a genetic algorithm and fuzzy markup language to construct a human and smart machine cooperative learning system on game of Go. The genetic fuzzy markup language (GFML)-based Robot Agent can work on various kinds of robots, including Palro, Pepper, and TMUs robots. We use the parameters of FAIR open source Darkforest and OpenGo AI bots to construct the knowledge base of Open Go Darkforest (OGD) cloud platform for student learning on the Internet. In addition, we adopt the data from AlphaGo Master sixty online games as the training data to construct the knowledge base and rule base of the co-learning system. First, the Darkforest predicts the win rate based on various simulation numbers and matching rates for each game on OGD platform, then the win rate of OpenGo is as the final desired output. The experimental results show that the proposed approach can improve knowledge base and rule base of the prediction ability based on Darkforest and OpenGo AI bot with various simulation numbers. △ Less

Submitted 22 January, 2019; originally announced January 2019.

Explain to Fix: A Framework to Interpret and Correct DNN Object Detector Predictions

Authors: Denis Gudovskiy, Alec Hodgkinson, Takuya Yamaguchi, Yasunori Ishii, Sotaro Tsukizawa

Abstract: Explaining predictions of deep neural networks (DNNs) is an important and nontrivial task. In this paper, we propose a practical approach to interpret decisions made by a DNN object detector that has fidelity comparable to state-of-the-art methods and sufficient computational efficiency to process large datasets. Our method relies on recent theory and approximates Shapley feature importance values… ▽ More Explaining predictions of deep neural networks (DNNs) is an important and nontrivial task. In this paper, we propose a practical approach to interpret decisions made by a DNN object detector that has fidelity comparable to state-of-the-art methods and sufficient computational efficiency to process large datasets. Our method relies on recent theory and approximates Shapley feature importance values. We qualitatively and quantitatively show that the proposed explanation method can be used to find image features which cause failures in DNN object detection. The developed software tool combined into the "Explain to Fix" (E2X) framework has a factor of 10 higher computational efficiency than prior methods and can be used for cluster processing using graphics processing units (GPUs). Lastly, we propose a potential extension of the E2X framework where the discovered missing features can be added into training dataset to overcome failures after model retraining. △ Less

Submitted 19 November, 2018; originally announced November 2018.

Comments: Systems for ML Workshop @ NIPS 2018

A Proposal of Interactive Growing Hierarchical SOM

Authors: Takumi Ichimura, Takashi Yamaguchi

Abstract: Self Organizing Map is trained using unsupervised learning to produce a two-dimensional discretized representation of input space of the training cases. Growing Hierarchical SOM is an architecture which grows both in a hierarchical way representing the structure of data distribution and in a horizontal way representation the size of each individual maps. The control method of the growing degree of… ▽ More Self Organizing Map is trained using unsupervised learning to produce a two-dimensional discretized representation of input space of the training cases. Growing Hierarchical SOM is an architecture which grows both in a hierarchical way representing the structure of data distribution and in a horizontal way representation the size of each individual maps. The control method of the growing degree of GHSOM by pruning off the redundant branch of hierarchy in SOM is proposed in this paper. Moreover, the interface tool for the proposed method called interactive GHSOM is developed. We discuss the computation results of Iris data by using the developed tool. △ Less

Submitted 7 April, 2018; originally announced April 2018.

Comments: 6 pages, 11 figures, Proc. of 2011 IEEE International Conference on Systems, Man, and Cybernetics (IEEE SMC 2011). arXiv admin note: text overlap with arXiv:cs/0405030 by other authors

Implicit Low-Order Unstructured Finite-Element Multiple Simulation Enhanced by Dense Computation using OpenACC

Authors: Takuma Yamaguchi, Kohei Fujita, Tsuyoshi Ichimura, Muneo Hori, Maddegedara Lalith, Kengo Nakajima

Abstract: In this paper, we develop a low-order three-dimensional finite-element solver for fast multiple-case crust deformation analysis on GPU-based systems. Based on a high-performance solver designed for massively parallel CPU based systems, we modify the algorithm to reduce random data access, and then insert OpenACC directives. The developed solver on ten Reedbush-H nodes (20 P100 GPUs) attained speed… ▽ More In this paper, we develop a low-order three-dimensional finite-element solver for fast multiple-case crust deformation analysis on GPU-based systems. Based on a high-performance solver designed for massively parallel CPU based systems, we modify the algorithm to reduce random data access, and then insert OpenACC directives. The developed solver on ten Reedbush-H nodes (20 P100 GPUs) attained speedup of 14.2 times from 20 K computer nodes, which is high considering the peak memory bandwidth ratio of 11.4 between the two systems. On the newest Volta generation V100 GPUs, the solver attained a further 2.45 times speedup from P100 GPUs. As a demonstrative example, we computed 368 cases of crustal deformation analyses of northeast Japan with 400 million degrees of freedom. The total procedure of algorithm modification and porting implementation took only two weeks; we can see that high performance improvement was achieved with low development cost. With the developed solver, we can expect improvement in reliability of crust-deformation analyses by many-case analyses on a wide range of GPU-based systems. △ Less

Submitted 24 October, 2017; originally announced October 2017.

Comments: 18 pages, 10 figures, accepted for WACCPD2017

Proceedings of the Pacific Knowledge Acquisition Workshop 2004

Authors: Byeong Ho Kang, Achim Hoffmann, Takahira Yamaguchi, Wai Kiang Yeap

Abstract: Artificial intelligence (AI) research has evolved over the last few decades and knowledge acquisition research is at the core of AI research. PKAW-04 is one of three international knowledge acquisition workshops held in the Pacific-Rim, Canada and Europe over the last two decades. PKAW-04 has a strong emphasis on incremental knowledge acquisition, machine learning, neural nets and active mining.… ▽ More Artificial intelligence (AI) research has evolved over the last few decades and knowledge acquisition research is at the core of AI research. PKAW-04 is one of three international knowledge acquisition workshops held in the Pacific-Rim, Canada and Europe over the last two decades. PKAW-04 has a strong emphasis on incremental knowledge acquisition, machine learning, neural nets and active mining. The proceedings contain 19 papers that were selected by the program committee among 24 submitted papers. All papers were peer reviewed by at least two reviewers. The papers in these proceedings cover the methods and tools as well as the applications related to develop expert systems or knowledge based systems. △ Less

Submitted 14 April, 2005; originally announced April 2005.