Showing 1–25 of 25 results for author: Ikeda, K

Speed-accuracy trade-off for the diffusion models: Wisdom from nonequilibrium thermodynamics and optimal transport

Authors: Kotaro Ikeda, Tomoya Uda, Daisuke Okanohara, Sosuke Ito

Abstract: We discuss a connection between a generative model, called the diffusion model, and nonequilibrium thermodynamics for the Fokker-Planck equation, called stochastic thermodynamics. Based on the techniques of stochastic thermodynamics, we derive the speed-accuracy trade-off for the diffusion models, which is a trade-off relationship between the speed and accuracy of data generation in diffusion mode… ▽ More We discuss a connection between a generative model, called the diffusion model, and nonequilibrium thermodynamics for the Fokker-Planck equation, called stochastic thermodynamics. Based on the techniques of stochastic thermodynamics, we derive the speed-accuracy trade-off for the diffusion models, which is a trade-off relationship between the speed and accuracy of data generation in diffusion models. Our result implies that the entropy production rate in the forward process affects the errors in data generation. From a stochastic thermodynamic perspective, our results provide quantitative insight into how best to generate data in diffusion models. The optimal learning protocol is introduced by the conservative force in stochastic thermodynamics and the geodesic of space by the 2-Wasserstein distance in optimal transport theory. We numerically illustrate the validity of the speed-accuracy trade-off for the diffusion models with different noise schedules such as the cosine schedule, the conditional optimal transport, and the optimal transport. △ Less

Submitted 7 July, 2024; v1 submitted 5 July, 2024; originally announced July 2024.

Comments: 26 pages, 5 figures

Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information Retrieval

Authors: Kazuaki Furumai, Roberto Legaspi, Julio Vizcarra, Yudai Yamazaki, Yasutaka Nishimura, Sina J. Semnani, Kazushi Ikeda, Weiyan Shi, Monica S. Lam

Abstract: Persuasion plays a pivotal role in a wide range of applications from health intervention to the promotion of social good. Persuasive chatbots can accelerate the positive effects of persuasion in such applications. Existing methods rely on fine-tuning persuasive chatbots with task-specific training data which is costly, if not infeasible, to collect. To address this issue, we propose a method to le… ▽ More Persuasion plays a pivotal role in a wide range of applications from health intervention to the promotion of social good. Persuasive chatbots can accelerate the positive effects of persuasion in such applications. Existing methods rely on fine-tuning persuasive chatbots with task-specific training data which is costly, if not infeasible, to collect. To address this issue, we propose a method to leverage the generalizability and inherent persuasive abilities of large language models (LLMs) in creating effective and truthful persuasive chatbot for any given domain in a zero-shot manner. Unlike previous studies which used pre-defined persuasion strategies, our method first uses an LLM to generate responses, then extracts the strategies used on the fly, and replaces any unsubstantiated claims in the response with retrieved facts supporting the strategies. We applied our chatbot, PersuaBot, to three significantly different domains needing persuasion skills: donation solicitation, recommendations, and health intervention. Our experiments on simulated and human conversations show that our zero-shot approach is more persuasive than prior work, while achieving factual accuracy surpassing state-of-the-art knowledge-oriented chatbots. Our study demonstrated that when persuasive chatbots are employed responsibly for social good, it is an enabler of positive individual and social change. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Counterfactual Reasoning Using Predicted Latent Personality Dimensions for Optimizing Persuasion Outcome

Authors: Donghuo Zeng, Roberto S. Legaspi, Yuewen Sun, Xinshuai Dong, Kazushi Ikeda, Peter Spirtes, kun Zhang

Abstract: Customizing persuasive conversations related to the outcome of interest for specific users achieves better persuasion results. However, existing persuasive conversation systems rely on persuasive strategies and encounter challenges in dynamically adjusting dialogues to suit the evolving states of individual users during interactions. This limitation restricts the system's ability to deliver flexib… ▽ More Customizing persuasive conversations related to the outcome of interest for specific users achieves better persuasion results. However, existing persuasive conversation systems rely on persuasive strategies and encounter challenges in dynamically adjusting dialogues to suit the evolving states of individual users during interactions. This limitation restricts the system's ability to deliver flexible or dynamic conversations and achieve suboptimal persuasion outcomes. In this paper, we present a novel approach that tracks a user's latent personality dimensions (LPDs) during ongoing persuasion conversation and generates tailored counterfactual utterances based on these LPDs to optimize the overall persuasion outcome. In particular, our proposed method leverages a Bi-directional Generative Adversarial Network (BiCoGAN) in tandem with a Dialogue-based Personality Prediction Regression (DPPR) model to generate counterfactual data. This enables the system to formulate alternative persuasive utterances that are more suited to the user. Subsequently, we utilize the D3QN model to learn policies for optimized selection of system utterances on counterfactual data. Experimental results we obtained from using the PersuasionForGood dataset demonstrate the superiority of our approach over the existing method, BiCoGAN. The cumulative rewards and Q-values produced by our method surpass ground truth benchmarks, showcasing the efficacy of employing counterfactual reasoning and LPDs to optimize reinforcement learning policy in online interactions. △ Less

Submitted 21 April, 2024; originally announced April 2024.

Comments: 14 pages, 10 figures, Accepted by Persuasive Technology 2024

Anchor-aware Deep Metric Learning for Audio-visual Retrieval

Authors: Donghuo Zeng, Yanan Wang, Kazushi Ikeda, Yi Yu

Abstract: Metric learning minimizes the gap between similar (positive) pairs of data points and increases the separation of dissimilar (negative) pairs, aiming at capturing the underlying data structure and enhancing the performance of tasks like audio-visual cross-modal retrieval (AV-CMR). Recent works employ sampling methods to select impactful data points from the embedding space during training. However… ▽ More Metric learning minimizes the gap between similar (positive) pairs of data points and increases the separation of dissimilar (negative) pairs, aiming at capturing the underlying data structure and enhancing the performance of tasks like audio-visual cross-modal retrieval (AV-CMR). Recent works employ sampling methods to select impactful data points from the embedding space during training. However, the model training fails to fully explore the space due to the scarcity of training data points, resulting in an incomplete representation of the overall positive and negative distributions. In this paper, we propose an innovative Anchor-aware Deep Metric Learning (AADML) method to address this challenge by uncovering the underlying correlations among existing data points, which enhances the quality of the shared embedding space. Specifically, our method establishes a correlation graph-based manifold structure by considering the dependencies between each sample as the anchor and its semantically similar samples. Through dynamic weighting of the correlations within this underlying manifold structure using an attention-driven mechanism, Anchor Awareness (AA) scores are obtained for each anchor. These AA scores serve as data proxies to compute relative distances in metric learning approaches. Extensive experiments conducted on two audio-visual benchmark datasets demonstrate the effectiveness of our proposed AADML method, significantly surpassing state-of-the-art models. Furthermore, we investigate the integration of AA proxies with various metric learning methods, further highlighting the efficacy of our approach. △ Less

Submitted 21 April, 2024; originally announced April 2024.

Comments: 9 pages, 5 figures. Accepted by ACM ICMR 2024

Contextualized Messages Boost Graph Representations

Authors: Brian Godwin Lim, Galvin Brice Lim, Renzo Roel Tan, Kazushi Ikeda

Abstract: Graph neural networks (GNNs) have gained significant attention in recent years for their ability to process data that may be represented as graphs. This success has prompted several studies to explore the representational capability of GNNs based on the graph isomorphism task. These works inherently assume a countable node feature representation, potentially limiting their applicability. Interesti… ▽ More Graph neural networks (GNNs) have gained significant attention in recent years for their ability to process data that may be represented as graphs. This success has prompted several studies to explore the representational capability of GNNs based on the graph isomorphism task. These works inherently assume a countable node feature representation, potentially limiting their applicability. Interestingly, only a few theoretical works study GNNs with uncountable node feature representation. This paper presents a novel perspective on the representational capability of GNNs across all levels - node-level, neighborhood-level, and graph-level - when the space of node feature representation is uncountable. Specifically, it relaxes the injective requirement in previous works by employing an implicit pseudometric distance on the space of input to create a soft-injective function. This allows distinct inputs to produce similar outputs only if the pseudometric deems the inputs to be sufficiently similar on some representation, which is often useful in practice. As a consequence, a novel soft-isomorphic relational graph convolution network (SIR-GCN) that emphasizes non-linear and contextualized transformation of neighborhood feature representations is proposed. A mathematical discussion on the relationship between SIR-GCN and widely used GNNs is then laid out to put the contribution in context, establishing SIR-GCN as a generalization of classical GNN methodologies. Experiments on synthetic and benchmark datasets demonstrate the relative superiority of SIR-GCN, outperforming comparable models in node and graph property prediction tasks. △ Less

Submitted 22 May, 2024; v1 submitted 19 March, 2024; originally announced March 2024.

Symmetric silicon microring resonator optical crossbar array for accelerated inference and training in deep learning

Authors: Rui Tang, Shuhei Ohno, Ken Tanizawa, Kazuhiro Ikeda, Makoto Okano, Kasidit Toprasertpong, Shinichi Takagi, Mitsuru Takenaka

Abstract: Photonic integrated circuits are emerging as a promising platform for accelerating matrix multiplications in deep learning, leveraging the inherent parallel nature of light. Although various schemes have been proposed and demonstrated to realize such photonic matrix accelerators, the in-situ training of artificial neural networks using photonic accelerators remains challenging due to the difficult… ▽ More Photonic integrated circuits are emerging as a promising platform for accelerating matrix multiplications in deep learning, leveraging the inherent parallel nature of light. Although various schemes have been proposed and demonstrated to realize such photonic matrix accelerators, the in-situ training of artificial neural networks using photonic accelerators remains challenging due to the difficulty of direct on-chip backpropagation on a photonic chip. In this work, we propose a silicon microring resonator (MRR) optical crossbar array with a symmetric structure that allows for simple on-chip backpropagation, potentially enabling the acceleration of both the inference and training phases of deep learning. We demonstrate a $4 \times 4$ circuit on a Si-on-insulator (SOI) platform and use it to perform inference tasks of a simple neural network for classifying Iris flowers, achieving a classification accuracy of 93.3%. Subsequently, we train the neural network using simulated on-chip backpropagation and achieve an accuracy of 91.1% in the same inference task after training. Furthermore, we simulate a convolutional neural network (CNN) for handwritten digit recognition, using a $9 \times 9$ MRR crossbar array to perform the convolution operations. This work contributes to the realization of compact and energy-efficient photonic accelerators for deep learning. △ Less

Submitted 1 June, 2024; v1 submitted 29 January, 2024; originally announced January 2024.

Journal ref: Photonics Research, 2024

Quantum-classical simulation of quantum field theory by quantum circuit learning

Authors: Kazuki Ikeda

Abstract: We employ quantum circuit learning to simulate quantum field theories (QFTs). Typically, when simulating QFTs with quantum computers, we encounter significant challenges due to the technical limitations of quantum devices when implementing the Hamiltonian using Pauli spin matrices. To address this challenge, we leverage quantum circuit learning, employing a compact configuration of qubits and low-… ▽ More We employ quantum circuit learning to simulate quantum field theories (QFTs). Typically, when simulating QFTs with quantum computers, we encounter significant challenges due to the technical limitations of quantum devices when implementing the Hamiltonian using Pauli spin matrices. To address this challenge, we leverage quantum circuit learning, employing a compact configuration of qubits and low-depth quantum circuits to predict real-time dynamics in quantum field theories. The key advantage of this approach is that a single-qubit measurement can accurately forecast various physical parameters, including fully-connected operators. To demonstrate the effectiveness of our method, we use it to predict quench dynamics, chiral dynamics and jet production in a 1+1-dimensional model of quantum electrodynamics. We find that our predictions closely align with the results of rigorous classical calculations, exhibiting a high degree of accuracy. This hybrid quantum-classical approach illustrates the feasibility of efficiently simulating large-scale QFTs on cutting-edge quantum devices. △ Less

Submitted 27 November, 2023; originally announced November 2023.

Two-Stage Triplet Loss Training with Curriculum Augmentation for Audio-Visual Retrieval

Authors: Donghuo Zeng, Kazushi Ikeda

Abstract: The cross-modal retrieval model leverages the potential of triple loss optimization to learn robust embedding spaces. However, existing methods often train these models in a singular pass, overlooking the distinction between semi-hard and hard triples in the optimization process. The oversight of not distinguishing between semi-hard and hard triples leads to suboptimal model performance. In this p… ▽ More The cross-modal retrieval model leverages the potential of triple loss optimization to learn robust embedding spaces. However, existing methods often train these models in a singular pass, overlooking the distinction between semi-hard and hard triples in the optimization process. The oversight of not distinguishing between semi-hard and hard triples leads to suboptimal model performance. In this paper, we introduce a novel approach rooted in curriculum learning to address this problem. We propose a two-stage training paradigm that guides the model's learning process from semi-hard to hard triplets. In the first stage, the model is trained with a set of semi-hard triplets, starting from a low-loss base. Subsequently, in the second stage, we augment the embeddings using an interpolation technique. This process identifies potential hard negatives, alleviating issues arising from high-loss functions due to a scarcity of hard triples. Our approach then applies hard triplet mining in the augmented embedding space to further optimize the model. Extensive experimental results conducted on two audio-visual datasets show a significant improvement of approximately 9.8% in terms of average Mean Average Precision (MAP) over the current state-of-the-art method, MSNSCA, for the Audio-Visual Cross-Modal Retrieval (AV-CMR) task on the AVE dataset, indicating the effectiveness of our proposed method. △ Less

Submitted 20 October, 2023; originally announced October 2023.

Comments: 8 pages, 6 figures

Quantum interactive proofs using quantum energy teleportation

Authors: Kazuki Ikeda, Adam Lowe

Abstract: We present a simple quantum interactive proof (QIP) protocol using the quantum state teleportation (QST) and quantum energy teleportation (QET) protocols. QET is a technique that allows a receiver at a distance to extract the local energy by local operations and classical communication (LOCC), using the energy injected by the supplier as collateral. QET works for any local Hamiltonian with entangl… ▽ More We present a simple quantum interactive proof (QIP) protocol using the quantum state teleportation (QST) and quantum energy teleportation (QET) protocols. QET is a technique that allows a receiver at a distance to extract the local energy by local operations and classical communication (LOCC), using the energy injected by the supplier as collateral. QET works for any local Hamiltonian with entanglement and, for our study, it is important that getting the ground state of a generic local Hamiltonian is quantum Merlin Arthur (QMA)-hard. The key motivations behind employing QET for these purposes are clarified. Firstly, in cases where a prover possesses the correct state and executes the appropriate operations, the verifier can effectively validate the presence of negative energy with a high probability (Completeness). Failure to select the appropriate operators or an incorrect state renders the verifier incapable of observing negative energy (Soundness). Importantly, the verifier solely observes a single qubit from the prover's transmitted state, while remaining oblivious to the prover's Hamiltonian and state (Zero-knowledge). Furthermore, the analysis is extended to distributed quantum interactive proofs, where we propose multiple solutions for the verification of each player's measurement. The complexity class of our protocol in the most general case belongs to QIP(3)=PSPACE, hence it provides a secure quantum authentication scheme that can be implemented in small quantum communication devices. It is straightforward to extend our protocol to Quantum Multi-Prover Interactive Proof (QMIP) systems, where the complexity is expected to be more powerful (PSPACE$\subset$QMIP=NEXPTIME). In our case, all provers share the ground state entanglement, hence it should belong to a more powerful complexity class QMIP$^*$. △ Less

Submitted 14 June, 2023; originally announced June 2023.

Journal ref: Quantum Information Processing 23, 236 (2024)

Natural Evolution Strategy for Mixed-Integer Black-Box Optimization

Authors: Koki Ikeda, Isao Ono

Abstract: This paper proposes a natural evolution strategy (NES) for mixed-integer black-box optimization (MI-BBO) that appears in real-world problems such as hyperparameter optimization of machine learning and materials design. This problem is difficult to optimize because plateaus where the values do not change appear when the integer variables are relaxed to the continuous ones. CMA-ES w. Margin that add… ▽ More This paper proposes a natural evolution strategy (NES) for mixed-integer black-box optimization (MI-BBO) that appears in real-world problems such as hyperparameter optimization of machine learning and materials design. This problem is difficult to optimize because plateaus where the values do not change appear when the integer variables are relaxed to the continuous ones. CMA-ES w. Margin that addresses the plateaus reportedly showed good performance on MI-BBO benchmark problems. However, it has been observed that the search performance of CMA-ES w. Margin deteriorates when continuous variables contribute more to the objective function value than integer ones. In order to address the problem of CMA-ES w. Margin, we propose Distance-weighted eXponential Natural Evolution Strategy taking account of Implicit Constraint and Integer (DX-NES-ICI). We compare the search performance of DX-NES-ICI with that of CMA-ES w. Margin through numerical experiments. As a result, DX-NES-ICI was up to 3.7 times better than CMA-ES w. Margin in terms of a rate of finding the optimal solutions on benchmark problems where continuous variables contribute more to the objective function value than integer ones. DX-NES-ICI also outperformed CMA-ES w. Margin on problems where CMA-ES w. Margin originally showed good performance. △ Less

Submitted 20 April, 2023; originally announced April 2023.

Comments: Accepted for GECCO'23

Topic-switch adapted Japanese Dialogue System based on PLATO-2

Authors: Donghuo Zeng, Jianming Wu, Yanan Wang, Kazunori Matsumoto, Gen Hattori, Kazushi Ikeda

Abstract: Large-scale open-domain dialogue systems such as PLATO-2 have achieved state-of-the-art scores in both English and Chinese. However, little work explores whether such dialogue systems also work well in the Japanese language. In this work, we create a large-scale Japanese dialogue dataset, Dialogue-Graph, which contains 1.656 million dialogue data in a tree structure from News, TV subtitles, and Wi… ▽ More Large-scale open-domain dialogue systems such as PLATO-2 have achieved state-of-the-art scores in both English and Chinese. However, little work explores whether such dialogue systems also work well in the Japanese language. In this work, we create a large-scale Japanese dialogue dataset, Dialogue-Graph, which contains 1.656 million dialogue data in a tree structure from News, TV subtitles, and Wikipedia corpus. Then, we train PLATO-2 using Dialogue-Graph to build a large-scale Japanese dialogue system, PLATO-JDS. In addition, to improve the PLATO-JDS in the topic switch issue, we introduce a topic-switch algorithm composed of a topic discriminator to switch to a new topic when user input differs from the previous topic. We evaluate the user experience by using our model with respect to four metrics, namely, coherence, informativeness, engagingness, and humanness. As a result, our proposed PLATO-JDS achieves an average score of 1.500 for the human evaluation with human-bot chat strategy, which is close to the maximum score of 2.000 and suggests the high-quality dialogue generation capability of PLATO-2 in Japanese. Furthermore, our proposed topic-switch algorithm achieves an average score of 1.767 and outperforms PLATO-JDS by 0.267, indicating its effectiveness in improving the user experience of our system. △ Less

Submitted 22 February, 2023; originally announced February 2023.

Comments: 10 pages, 8 figures, 7 tables

Do I Have Your Attention: A Large Scale Engagement Prediction Dataset and Baselines

Authors: Monisha Singh, Ximi Hoque, Donghuo Zeng, Yanan Wang, Kazushi Ikeda, Abhinav Dhall

Abstract: The degree of concentration, enthusiasm, optimism, and passion displayed by individual(s) while interacting with a machine is referred to as `user engagement'. Engagement comprises of behavioral, cognitive, and affect related cues. To create engagement prediction systems that can work in real-world conditions, it is quintessential to learn from rich, diverse datasets. To this end, a large scale mu… ▽ More The degree of concentration, enthusiasm, optimism, and passion displayed by individual(s) while interacting with a machine is referred to as `user engagement'. Engagement comprises of behavioral, cognitive, and affect related cues. To create engagement prediction systems that can work in real-world conditions, it is quintessential to learn from rich, diverse datasets. To this end, a large scale multi-faceted engagement in the wild dataset EngageNet is proposed. 31 hours duration data of 127 participants representing different illumination conditions are recorded. Thorough experiments are performed exploring the applicability of different features, action units, eye gaze, head pose, and MARLIN. Data from user interactions (question-answer) are analyzed to understand the relationship between effective learning and user engagement. To further validate the rich nature of the dataset, evaluation is also performed on the EngageWild dataset. The experiments show the usefulness of the proposed dataset. The code, models, and dataset link are publicly available at https://github.com/engagenet/engagenet_baselines. △ Less

Submitted 17 August, 2023; v1 submitted 1 February, 2023; originally announced February 2023.

Long-range quantum energy teleportation and distribution on a hyperbolic quantum network

Authors: Kazuki Ikeda

Abstract: Teleporting energy to remote locations is new challenge for quantum information science and technology. Developing a method for transferring local energy in laboratory systems to remote locations will enable non-trivial energy flows in quantum networks. From the perspective of quantum information engineering, we propose a method for distributing local energy to a large number of remote nodes using… ▽ More Teleporting energy to remote locations is new challenge for quantum information science and technology. Developing a method for transferring local energy in laboratory systems to remote locations will enable non-trivial energy flows in quantum networks. From the perspective of quantum information engineering, we propose a method for distributing local energy to a large number of remote nodes using hyperbolic geometry. Hyperbolic networks are suitable for energy allocation in large quantum networks since the number of nodes grows exponentially. To realise long-range quantum energy teleportation, we propose a hybrid method of quantum state telepotation and quantum energy teleportation. By transmitting local quantum information through quantum teleportation and performing conditional operations on that information, quantum energy teleportation can theoretically be realized independent of geographical distance. The method we present will provide new insights into new applications of future large-scale quantum networks and potential applications of quantum physics to information engineering. △ Less

Submitted 27 January, 2023; originally announced January 2023.

Journal ref: IET Quant. Comm. 1-8 (2024)

Complete Cross-triplet Loss in Label Space for Audio-visual Cross-modal Retrieval

Authors: Donghuo Zeng, Yanan Wang, Jianming Wu, Kazushi Ikeda

Abstract: The heterogeneity gap problem is the main challenge in cross-modal retrieval. Because cross-modal data (e.g. audiovisual) have different distributions and representations that cannot be directly compared. To bridge the gap between audiovisual modalities, we learn a common subspace for them by utilizing the intrinsic correlation in the natural synchronization of audio-visual data with the aid of an… ▽ More The heterogeneity gap problem is the main challenge in cross-modal retrieval. Because cross-modal data (e.g. audiovisual) have different distributions and representations that cannot be directly compared. To bridge the gap between audiovisual modalities, we learn a common subspace for them by utilizing the intrinsic correlation in the natural synchronization of audio-visual data with the aid of annotated labels. TNN-CCCA is the best audio-visual cross-modal retrieval (AV-CMR) model so far, but the model training is sensitive to hard negative samples when learning common subspace by applying triplet loss to predict the relative distance between inputs. In this paper, to reduce the interference of hard negative samples in representation learning, we propose a new AV-CMR model to optimize semantic features by directly predicting labels and then measuring the intrinsic correlation between audio-visual data using complete cross-triple loss. In particular, our model projects audio-visual features into label space by minimizing the distance between predicted label features after feature projection and ground label representations. Moreover, we adopt complete cross-triplet loss to optimize the predicted label features by leveraging the relationship between all possible similarity and dissimilarity semantic information across modalities. The extensive experimental results on two audio-visual double-checked datasets have shown an improvement of approximately 2.1% in terms of average MAP over the current state-of-the-art method TNN-CCCA for the AV-CMR task, which indicates the effectiveness of our proposed model. △ Less

Submitted 7 November, 2022; originally announced November 2022.

Comments: 9 pages, 5 figures, 3 tables, accepted by IEEE ISM 2022

Quantum Protocol for Decision Making and Verifying Truthfulness among $N$-quantum Parties: Solution and Extension of the Quantum Coin Flipping Game

Authors: Kazuki Ikeda, Adam Lowe

Abstract: We devised a protocol that allows two parties, who may malfunction or intentionally convey incorrect information in communication through a quantum channel, to verify each other's measurements and agree on each other's results. This has particular relevance in a modified version of the quantum coin flipping game where the possibility of the players cheating is now removed. Furthermore, the analysi… ▽ More We devised a protocol that allows two parties, who may malfunction or intentionally convey incorrect information in communication through a quantum channel, to verify each other's measurements and agree on each other's results. This has particular relevance in a modified version of the quantum coin flipping game where the possibility of the players cheating is now removed. Furthermore, the analysis is extended to $N$-parties communicating with each other, where we propose multiple solutions for the verification of each player's measurement. The results in the $N$-party scenario could have particular relevance for the implementation of future quantum networks, where verification of quantum information is a necessity. △ Less

Submitted 3 November, 2022; originally announced November 2022.

Journal ref: IET Quant. Comm. 1-10 (2023)

Compositionality-Aware Graph2Seq Learning

Authors: Takeshi D. Itoh, Takatomi Kubo, Kazushi Ikeda

Abstract: Graphs are a highly expressive data structure, but it is often difficult for humans to find patterns from a complex graph. Hence, generating human-interpretable sequences from graphs have gained interest, called graph2seq learning. It is expected that the compositionality in a graph can be associated to the compositionality in the output sequence in many graph2seq tasks. Therefore, applying compos… ▽ More Graphs are a highly expressive data structure, but it is often difficult for humans to find patterns from a complex graph. Hence, generating human-interpretable sequences from graphs have gained interest, called graph2seq learning. It is expected that the compositionality in a graph can be associated to the compositionality in the output sequence in many graph2seq tasks. Therefore, applying compositionality-aware GNN architecture would improve the model performance. In this study, we adopt the multi-level attention pooling (MLAP) architecture, that can aggregate graph representations from multiple levels of information localities. As a real-world example, we take up the extreme source code summarization task, where a model estimate the name of a program function from its source code. We demonstrate that the model having the MLAP architecture outperform the previous state-of-the-art model with more than seven times fewer parameters than it. △ Less

Submitted 28 January, 2022; originally announced January 2022.

Comments: 8 pages, 1 figure, 2 tables

Reduced-Lead ECG Classifier Model Trained with DivideMix and Model Ensemble

Authors: Hiroshi Seki, Takashi Nakano, Koshiro Ikeda, Shinji Hirooka, Takaaki Kawasaki, Mitsutomo Yamada, Shumpei Saito, Toshitaka Yamakawa, Shimpei Ogawa

Abstract: Automatic diagnosis of multiple cardiac abnormalities from reduced-lead electrocardiogram (ECG) data is challenging. One of the reasons for this is the difficulty of defining labels from standard 12-lead data. Reduced-lead ECG data usually do not have identical characteristics of cardiac abnormalities because of the noisy label problem. Thus, there is an inconsistency in the annotated labels betwe… ▽ More Automatic diagnosis of multiple cardiac abnormalities from reduced-lead electrocardiogram (ECG) data is challenging. One of the reasons for this is the difficulty of defining labels from standard 12-lead data. Reduced-lead ECG data usually do not have identical characteristics of cardiac abnormalities because of the noisy label problem. Thus, there is an inconsistency in the annotated labels between the reduced-lead and 12-lead ECG data. To solve this, we propose deep neural network (DNN)-based ECG classifier models that incorporate DivideMix and stochastic weight averaging (SWA). DivideMix was used to refine the noisy label by using two separate models. Besides DivideMix, we used a model ensemble technique, SWA, which also focuses on the noisy label problem, to enhance the effect of the models generated by DivideMix. Our classifiers (ami_kagoshima) received scores of 0.49, 0.47, 0.48, 0.47, and 0.47 (ranked 9th, 10th, 10th, 11th, and 10th, respectively, out of 39 teams) for the 12-lead, 6-lead, 4-lead, 3-lead, and 2-lead versions, respectively, of the hidden test set with the challenge evaluation metric. We obtained the scores of 0.701, 0.686, 0.693, 0.693, and 0.685 on the 10-fold cross validation, and 0.623, 0.593, 0.606, 0.612, and 0.601 on the hidden validation set for each lead combination. △ Less

Submitted 24 September, 2021; originally announced September 2021.

Comments: 4 pages, in Computing in Cardiology (CinC) 2021, PhysioNet/CinC Challenge 2021 submission

Multi-Level Attention Pooling for Graph Neural Networks: Unifying Graph Representations with Multiple Localities

Authors: Takeshi D. Itoh, Takatomi Kubo, Kazushi Ikeda

Abstract: Graph neural networks (GNNs) have been widely used to learn vector representation of graph-structured data and achieved better task performance than conventional methods. The foundation of GNNs is the message passing procedure, which propagates the information in a node to its neighbors. Since this procedure proceeds one step per layer, the range of the information propagation among nodes is small… ▽ More Graph neural networks (GNNs) have been widely used to learn vector representation of graph-structured data and achieved better task performance than conventional methods. The foundation of GNNs is the message passing procedure, which propagates the information in a node to its neighbors. Since this procedure proceeds one step per layer, the range of the information propagation among nodes is small in the lower layers, and it expands toward the higher layers. Therefore, a GNN model has to be deep enough to capture global structural information in a graph. On the other hand, it is known that deep GNN models suffer from performance degradation because they lose nodes' local information, which would be essential for good model performance, through many message passing steps. In this study, we propose multi-level attention pooling (MLAP) for graph-level classification tasks, which can adapt to both local and global structural information in a graph. It has an attention pooling layer for each message passing step and computes the final graph representation by unifying the layer-wise graph representations. The MLAP architecture allows models to utilize the structural information of graphs with multiple levels of localities because it preserves layer-wise information before losing them due to oversmoothing. Results of our experiments show that the MLAP architecture improves the graph classification performance compared to the baseline architectures. In addition, analyses on the layer-wise graph representations suggest that aggregating information from multiple levels of localities indeed has the potential to improve the discriminability of learned graph representations. △ Less

Submitted 31 October, 2021; v1 submitted 2 March, 2021; originally announced March 2021.

Comments: 17 pages + 5 appendix pages, 16 figures + 2 appendix figures, 2 tables + 5 appendix tables

Journal ref: Neural Netw. 145 (2022) 356-373

Theory of Quantum Games and Quantum Economic Behavior

Authors: Kazuki Ikeda, Shoto Aoki

Abstract: The quest of this work is to present discussions of some fundamental questions of economics in the era of quantum technology, which require a treatment different from economics studied thus far in the literature. A study of quantum economic behavior will become the center of attention of economists in the coming decades. We analyze a quantum economy in which players produce and consume quantum goo… ▽ More The quest of this work is to present discussions of some fundamental questions of economics in the era of quantum technology, which require a treatment different from economics studied thus far in the literature. A study of quantum economic behavior will become the center of attention of economists in the coming decades. We analyze a quantum economy in which players produce and consume quantum goods. They meet randomly and barter with neighbors bilaterally for quantum goods they produced. We clarify the conditions where certain quantum goods emerge endogenously as media of exchange, called quantum commodity money. As quantum strategies are entangled, we find distinctive aspects of quantum games that cannot be explained by conventional classical games. In some situations a quantum player can acquire a quantum good from people regardless of their strategies, while on the other hand people can find quantum strategies that improve their welfare based on an agreement. Those novel properties imply that quantum games also shed new light on theories of mechanism design, auction and contract in the quantum era. △ Less

Submitted 27 November, 2021; v1 submitted 27 October, 2020; originally announced October 2020.

Comments: 23 pages, 13 figures

Journal ref: Quantum Inf Process 21, 27 (2022)

Infinitely Repeated Quantum Games and Strategic Efficiency

Authors: Kazuki Ikeda, Shoto Aoki

Abstract: Repeated quantum game theory addresses long term relations among players who choose quantum strategies. In the conventional quantum game theory, single round quantum games or at most finitely repeated games have been widely studied, however less is known for infinitely repeated quantum games. Investigating infinitely repeated games is crucial since finitely repeated games do not much differ from s… ▽ More Repeated quantum game theory addresses long term relations among players who choose quantum strategies. In the conventional quantum game theory, single round quantum games or at most finitely repeated games have been widely studied, however less is known for infinitely repeated quantum games. Investigating infinitely repeated games is crucial since finitely repeated games do not much differ from single round games. In this work we establish the concept of general repeated quantum games and show the Quantum Folk Theorem, which claims that by iterating a game one can find an equilibrium strategy of the game and receive reward that is not obtained by a Nash equilibrium of the corresponding single round quantum game. A significant difference between repeated quantum prisoner's dilemma and repeated classical prisoner's dilemma is that the classical Pareto optimal solution is not always an equilibrium of the repeated quantum game when entanglement is sufficiently strong. When entanglement is sufficiently strong and reward is small, mutual cooperation cannot be an equilibrium of the repeated quantum game. In addition we present several concrete equilibrium strategies of the repeated quantum prisoner's dilemma. △ Less

Submitted 27 November, 2021; v1 submitted 12 May, 2020; originally announced May 2020.

Comments: 25 pages, 14 figures

Journal ref: Quantum Inf Process 20, 387 (2021)

Detecting Unknown Behaviors by Pre-defined Behaviours: An Bayesian Non-parametric Approach

Authors: Jin Watanabe, Takatomi Kubo, Fan Yang, Kazushi Ikeda

Abstract: An automatic mouse behavior recognition system can considerably reduce the workload of experimenters and facilitate the analysis process. Typically, supervised approaches, unsupervised approaches and semi-supervised approaches are applied for behavior recognition purpose under a setting which has all of predefined behaviors. In the real situation, however, as mouses can show various types of behav… ▽ More An automatic mouse behavior recognition system can considerably reduce the workload of experimenters and facilitate the analysis process. Typically, supervised approaches, unsupervised approaches and semi-supervised approaches are applied for behavior recognition purpose under a setting which has all of predefined behaviors. In the real situation, however, as mouses can show various types of behaviors, besides the predefined behaviors that we want to analyze, there are many undefined behaviors existing. Both supervised approaches and conventional semi-supervised approaches cannot identify these undefined behaviors. Though unsupervised approaches can detect these undefined behaviors, a post-hoc labeling is needed. In this paper, we propose a semi-supervised infinite Gaussian mixture model (SsIGMM), to incorporate both labeled and unlabelled information in learning process while considering undefined behaviors. It also generates the distribution of the predefined and undefined behaviors by mixture Gaussians, which can be used for further analysis. In our experiments, we confirmed the superiority of SsIGMM for segmenting and labelling mouse-behavior videos. △ Less

Submitted 11 December, 2019; v1 submitted 25 November, 2019; originally announced November 2019.

Analyzing Insect-Plant Predation Data By Bayesian Nonparametrics

Authors: Fan Yang, Takatomi Kubo, Kazushi Ikeda

Abstract: In the prospect of ecology and biology, studying insect-plant predation will considerably contribute to pest control, benefit agriculture and afforestation, and also help people to better understand insect-plant co-evolution. Therefore, we are motivated to do two work in this study. The first part is to cluster the insect-plant predation, in such manner, unobserved predation could be estimated. Th… ▽ More In the prospect of ecology and biology, studying insect-plant predation will considerably contribute to pest control, benefit agriculture and afforestation, and also help people to better understand insect-plant co-evolution. Therefore, we are motivated to do two work in this study. The first part is to cluster the insect-plant predation, in such manner, unobserved predation could be estimated. The second part is to explore the connection between predation and bio-taxonomy, and we find insects get more divergence than plants during the insect-plant co-evolution. △ Less

Submitted 11 December, 2019; v1 submitted 25 November, 2019; originally announced November 2019.

A Hierarchical Mixture Density Network

Authors: Fan Yang, Jaymar Soriano, Takatomi Kubo, Kazushi Ikeda

Abstract: The relationship among three correlated variables could be very sophisticated, as a result, we may not be able to find their hidden causality and model their relationship explicitly. However, we still can make our best guess for possible mappings among these variables, based on the observed relationship. One of the complicated relationships among three correlated variables could be a two-layer hie… ▽ More The relationship among three correlated variables could be very sophisticated, as a result, we may not be able to find their hidden causality and model their relationship explicitly. However, we still can make our best guess for possible mappings among these variables, based on the observed relationship. One of the complicated relationships among three correlated variables could be a two-layer hierarchical many-to-many mapping. In this paper, we proposed a Hierarchical Mixture Density Network (HMDN) to model the two-layer hierarchical many-to-many mapping. We apply HMDN on an indoor positioning problem and show its benefit. △ Less

Submitted 23 October, 2019; originally announced October 2019.

Comments: 8 pages, 5 figures, conference

Journal ref: The 24th International Conference on Neural Information Processing, 2017

Towards Generation of Visual Attention Map for Source Code

Authors: Takeshi D. Itoh, Takatomi Kubo, Kiyoka Ikeda, Yuki Maruno, Yoshiharu Ikutani, Hideaki Hata, Kenichi Matsumoto, Kazushi Ikeda

Abstract: Program comprehension is a dominant process in software development and maintenance. Experts are considered to comprehend the source code efficiently by directing their gaze, or attention, to important components in it. However, reflecting the importance of components is still a remaining issue in gaze behavior analysis for source code comprehension. Here we show a conceptual framework to compare… ▽ More Program comprehension is a dominant process in software development and maintenance. Experts are considered to comprehend the source code efficiently by directing their gaze, or attention, to important components in it. However, reflecting the importance of components is still a remaining issue in gaze behavior analysis for source code comprehension. Here we show a conceptual framework to compare the quantified importance of source code components with the gaze behavior of programmers. We use "attention" in attention models (e.g., code2vec) as the importance indices for source code components and evaluate programmers' gaze locations based on the quantified importance. In this report, we introduce the idea of our gaze behavior analysis using the attention map, and the results of a preliminary experiment. △ Less

Submitted 13 August, 2019; v1 submitted 14 July, 2019; originally announced July 2019.

Comments: 4 pages, 2 figures; APSIPA 2019 ACCEPTED

Journal ref: APSIPA ASC (2019)

qBitcoin: A Peer-to-Peer Quantum Cash System

Authors: Kazuki Ikeda

Abstract: A decentralized online quantum cash system, called qBitcoin, is given. We design the system which has great benefits of quantization in the following sense. Firstly, quantum teleportation technology is used for coin transaction, which prevents from the owner of the coin keeping the original coin data even after sending the coin to another. This was a main problem in a classical circuit and a block… ▽ More A decentralized online quantum cash system, called qBitcoin, is given. We design the system which has great benefits of quantization in the following sense. Firstly, quantum teleportation technology is used for coin transaction, which prevents from the owner of the coin keeping the original coin data even after sending the coin to another. This was a main problem in a classical circuit and a blockchain was introduced to solve this issue. In qBitcoin, the double-spending problem never happens and its security is guaranteed theoretically by virtue of quantum information theory. Making a block is time consuming and the system of qBitcoin is based on a quantum chain, instead of blocks. Therefore a payment can be completed much faster than Bitcoin. Moreover we employ quantum digital signature so that it naturally inherits properties of peer-to-peer (P2P) cash system as originally proposed in Bitcoin. △ Less

Submitted 19 November, 2017; v1 submitted 15 August, 2017; originally announced August 2017.

Comments: 11 pages, 2 figures

Report number: OU-HET-942

Journal ref: Proceedings of the 2018 Computing Conference