-
Human Mimetic Forearm Design with Radioulnar Joint using Miniature Bone-Muscle Modules and Its Applications
Authors:
Kento Kawaharazuka,
Shogo Makino,
Masaya Kawamura,
Yuki Asano,
Yohei Kakiuchi,
Kei Okada,
Masayuki Inaba
Abstract:
The human forearm is composed of two long, thin bones called the radius and the ulna, and rotates using two axle joints. We aimed to develop a forearm based on the body proportion, weight ratio, muscle arrangement, and joint performance of the human body in order to bring out its benefits. For this, we need to miniaturize the muscle modules. To approach this task, we arranged two muscle motors ins…
▽ More
The human forearm is composed of two long, thin bones called the radius and the ulna, and rotates using two axle joints. We aimed to develop a forearm based on the body proportion, weight ratio, muscle arrangement, and joint performance of the human body in order to bring out its benefits. For this, we need to miniaturize the muscle modules. To approach this task, we arranged two muscle motors inside one muscle module, and used the space effectively by utilizing common parts. In addition, we enabled the muscle module to also be used as the bone structure. Moreover, we used miniature motors and developed a way to dissipate the motor heat to the bone structure. Through these approaches, we succeeded in developing a forearm with a radioulnar joint based on the body proportion, weight ratio, muscle arrangement, and joint performance of the human body, while keeping maintainability and reliability. Also, we performed some motions such as soldering, opening a book, turning a screw, and badminton swinging using the benefits of the radioulnar structure, which have not been discussed before, and verified that Kengoro can realize skillful motions using the radioulnar joint like a human.
△ Less
Submitted 19 August, 2024;
originally announced August 2024.
-
A Method of Joint Angle Estimation Using Only Relative Changes in Muscle Lengths for Tendon-driven Humanoids with Complex Musculoskeletal Structures
Authors:
Kento Kawaharazuka,
Shogo Makino,
Masaya Kawamura,
Yuki Asano,
Kei Okada,
Masayuki Inaba
Abstract:
Tendon-driven musculoskeletal humanoids typically have complex structures similar to those of human beings, such as ball joints and the scapula, in which encoders cannot be installed. Therefore, joint angles cannot be directly obtained and need to be estimated using the changes in muscle lengths. In previous studies, methods using table-search and extended kalman filter have been developed. These…
▽ More
Tendon-driven musculoskeletal humanoids typically have complex structures similar to those of human beings, such as ball joints and the scapula, in which encoders cannot be installed. Therefore, joint angles cannot be directly obtained and need to be estimated using the changes in muscle lengths. In previous studies, methods using table-search and extended kalman filter have been developed. These methods express the joint-muscle mapping, which is the nonlinear relationship between joint angles and muscle lengths, by using a data table, polynomials, or a neural network. However, due to computational complexity, these methods cannot consider the effects of polyarticular muscles. In this study, considering the limitation of the computational cost, we reduce unnecessary degrees of freedom, divide joints and muscles into several groups, and formulate a joint angle estimation method that takes into account polyarticular muscles. Also, we extend the estimation method to propose a joint angle estimation method using only the relative changes in muscle lengths. By this extension, which does not use absolute muscle lengths, we do not need to execute a difficult calibration of muscle lengths for tendon-driven musculoskeletal humanoids. Finally, we conduct experiments in simulation and actual environments, and verify the effectiveness of this study.
△ Less
Submitted 22 April, 2024;
originally announced April 2024.
-
TWIMP: Two-Wheel Inverted Musculoskeletal Pendulum as a Learning Control Platform in the Real World with Environmental Physical Contact
Authors:
Kento Kawaharazuka,
Tasuku Makabe,
Shogo Makino,
Kei Tsuzuki,
Yuya Nagamatsu,
Yuki Asano,
Takuma Shirai,
Fumihito Sugai,
Kei Okada,
Koji Kawasaki,
Masayuki Inaba
Abstract:
By the recent spread of machine learning in the robotics field, a humanoid that can act, perceive, and learn in the real world through contact with the environment needs to be developed. In this study, as one of the choices, we propose a novel humanoid TWIMP, which combines a human mimetic musculoskeletal upper limb with a two-wheel inverted pendulum. By combining the benefit of a musculoskeletal…
▽ More
By the recent spread of machine learning in the robotics field, a humanoid that can act, perceive, and learn in the real world through contact with the environment needs to be developed. In this study, as one of the choices, we propose a novel humanoid TWIMP, which combines a human mimetic musculoskeletal upper limb with a two-wheel inverted pendulum. By combining the benefit of a musculoskeletal humanoid, which can achieve soft contact with the external environment, and the benefit of a two-wheel inverted pendulum with a small footprint and high mobility, we can easily investigate learning control systems in environments with contact and sudden impact. We reveal our whole concept and system details of TWIMP, and execute several preliminary experiments to show its potential ability.
△ Less
Submitted 22 April, 2024;
originally announced April 2024.
-
Online Learning of Joint-Muscle Mapping Using Vision in Tendon-driven Musculoskeletal Humanoids
Authors:
Kento Kawaharazuka,
Shogo Makino,
Masaya Kawamura,
Yuki Asano,
Kei Okada,
Masayuki Inaba
Abstract:
The body structures of tendon-driven musculoskeletal humanoids are complex, and accurate modeling is difficult, because they are made by imitating the body structures of human beings. For this reason, we have not been able to move them accurately like ordinary humanoids driven by actuators in each axis, and large internal muscle tension and slack of tendon wires have emerged by the model error bet…
▽ More
The body structures of tendon-driven musculoskeletal humanoids are complex, and accurate modeling is difficult, because they are made by imitating the body structures of human beings. For this reason, we have not been able to move them accurately like ordinary humanoids driven by actuators in each axis, and large internal muscle tension and slack of tendon wires have emerged by the model error between its geometric model and the actual robot. Therefore, we construct a joint-muscle mapping (JMM) using a neural network (NN), which expresses a nonlinear relationship between joint angles and muscle lengths, and aim to move tendon-driven musculoskeletal humanoids accurately by updating the JMM online from data of the actual robot. In this study, the JMM is updated online by using the vision of the robot so that it moves to the correct position (Vision Updater). Also, we execute another update to modify muscle antagonisms correctly (Antagonism Updater). By using these two updaters, the error between the target and actual joint angles decrease to about 40% in 5 minutes, and we show through a manipulation experiment that the tendon-driven musculoskeletal humanoid Kengoro becomes able to move as intended. This novel system can adapt to the state change and growth of robots, because it updates the JMM online successively.
△ Less
Submitted 8 April, 2024;
originally announced April 2024.
-
Long-time Self-body Image Acquisition and its Application to the Control of Musculoskeletal Structures
Authors:
Kento Kawaharazuka,
Kei Tsuzuki,
Shogo Makino,
Moritaka Onitsuka,
Yuki Asano,
Kei Okada,
Koji Kawasaki,
Masayuki Inaba
Abstract:
The tendon-driven musculoskeletal humanoid has many benefits that human beings have, but the modeling of its complex muscle and bone structures is difficult and conventional model-based controls cannot realize intended movements. Therefore, a learning control mechanism that acquires nonlinear relationships between joint angles, muscle tensions, and muscle lengths from the actual robot is necessary…
▽ More
The tendon-driven musculoskeletal humanoid has many benefits that human beings have, but the modeling of its complex muscle and bone structures is difficult and conventional model-based controls cannot realize intended movements. Therefore, a learning control mechanism that acquires nonlinear relationships between joint angles, muscle tensions, and muscle lengths from the actual robot is necessary. In this study, we propose a system which runs the learning control mechanism for a long time to keep the self-body image of the musculoskeletal humanoid correct at all times. Also, we show that the musculoskeletal humanoid can conduct position control, torque control, and variable stiffness control using this self-body image. We conduct a long-time self-body image acquisition experiment lasting 3 hours, evaluate variable stiffness control using the self-body image, etc., and discuss the superiority and practicality of the self-body image acquisition of musculoskeletal structures, comprehensively.
△ Less
Submitted 8 April, 2024;
originally announced April 2024.
-
Online Self-body Image Acquisition Considering Changes in Muscle Routes Caused by Softness of Body Tissue for Tendon-driven Musculoskeletal Humanoids
Authors:
Kento Kawaharazuka,
Shogo Makino,
Masaya Kawamura,
Ayaka Fujii,
Yuki Asano,
Kei Okada,
Masayuki Inaba
Abstract:
Tendon-driven musculoskeletal humanoids have many benefits in terms of the flexible spine, multiple degrees of freedom, and variable stiffness. At the same time, because of its body complexity, there are problems in controllability. First, due to the large difference between the actual robot and its geometric model, it cannot move as intended and large internal muscle tension may emerge. Second, m…
▽ More
Tendon-driven musculoskeletal humanoids have many benefits in terms of the flexible spine, multiple degrees of freedom, and variable stiffness. At the same time, because of its body complexity, there are problems in controllability. First, due to the large difference between the actual robot and its geometric model, it cannot move as intended and large internal muscle tension may emerge. Second, movements which do not appear as changes in muscle lengths may emerge, because of the muscle route changes caused by softness of body tissue. To solve these problems, we construct two models: ideal joint-muscle model and muscle-route change model, using a neural network. We initialize these models by a man-made geometric model and update them online using the sensor information of the actual robot. We validate that the tendon-driven musculoskeletal humanoid Kengoro is able to obtain a correct self-body image through several experiments.
△ Less
Submitted 8 April, 2024;
originally announced April 2024.
-
High-Power, Flexible, Robust Hand: Development of Musculoskeletal Hand Using Machined Springs and Realization of Self-Weight Supporting Motion with Humanoid
Authors:
Shogo Makino,
Kento Kawaharazuka,
Masaya Kawamura,
Yuki Asano,
Kei Okada,
Masayuki Inaba
Abstract:
Human can not only support their body during standing or walking, but also support them by hand, so that they can dangle a bar and others. But most humanoid robots support their body only in the foot and they use their hand just to manipulate objects because their hands are too weak to support their body. Strong hands are supposed to enable humanoid robots to act in much broader scene. Therefore,…
▽ More
Human can not only support their body during standing or walking, but also support them by hand, so that they can dangle a bar and others. But most humanoid robots support their body only in the foot and they use their hand just to manipulate objects because their hands are too weak to support their body. Strong hands are supposed to enable humanoid robots to act in much broader scene. Therefore, we developed new life-size five-fingered hand that can support the body of life-size humanoid robot. It is tendon-driven and underactuated hand and actuators in forearms produce large gripping force. This hand has flexible joints using machined springs, which can be designed integrally with the attachment. Thus, it has both structural strength and impact resistance in spite of small size. As other characteristics, this hand has force sensors to measure external force and the fingers can be flexed along objects though the number of actuators to flex fingers is less than that of fingers. We installed the developed hand on musculoskeletal humanoid "Kengoro" and achieved two self-weight supporting motions: push-up motion and dangling motion.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
Five-fingered Hand with Wide Range of Thumb Using Combination of Machined Springs and Variable Stiffness Joints
Authors:
Shogo Makino,
Kento Kawaharazuka,
Ayaka Fujii,
Masaya Kawamura,
Tasuku Makabe,
Moritaka Onitsuka,
Yuki Asano,
Kei Okada,
Koji Kawasaki,
Masayuki Inaba
Abstract:
Human hands can not only grasp objects of various shape and size and manipulate them in hands but also exert such a large gripping force that they can support the body in the situations such as dangling a bar and climbing a ladder. On the other hand, it is difficult for most robot hands to manage both. Therefore in this paper we developed the hand which can grasp various objects and exert large gr…
▽ More
Human hands can not only grasp objects of various shape and size and manipulate them in hands but also exert such a large gripping force that they can support the body in the situations such as dangling a bar and climbing a ladder. On the other hand, it is difficult for most robot hands to manage both. Therefore in this paper we developed the hand which can grasp various objects and exert large gripping force. To develop such hand, we focused on the thumb CM joint with wide range of motion and the MP joints of four fingers with the DOF of abduction and adduction. Based on the hand with large gripping force and flexibility using machined spring, we applied above mentioned joint mechanism to the hand. The thumb CM joint has wide range of motion because of the combination of three machined springs and MP joints of four fingers have variable rigidity mechanism instead of driving each joint independently in order to move joint in limited space and by limited actuators. Using the developed hand, we achieved the grasping of various objects, supporting a large load and several motions with an arm.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
Unrestricted Global Phase Bias-Aware Single-channel Speech Enhancement with Conformer-based Metric GAN
Authors:
Shiqi Zhang,
Zheng Qiu,
Daiki Takeuchi,
Noboru Harada,
Shoji Makino
Abstract:
With the rapid development of neural networks in recent years, the ability of various networks to enhance the magnitude spectrum of noisy speech in the single-channel speech enhancement domain has become exceptionally outstanding. However, enhancing the phase spectrum using neural networks is often ineffective, which remains a challenging problem. In this paper, we found that the human ear cannot…
▽ More
With the rapid development of neural networks in recent years, the ability of various networks to enhance the magnitude spectrum of noisy speech in the single-channel speech enhancement domain has become exceptionally outstanding. However, enhancing the phase spectrum using neural networks is often ineffective, which remains a challenging problem. In this paper, we found that the human ear cannot sensitively perceive the difference between a precise phase spectrum and a biased phase (BP) spectrum. Therefore, we propose an optimization method of phase reconstruction, allowing freedom on the global-phase bias instead of reconstructing the precise phase spectrum. We applied it to a Conformer-based Metric Generative Adversarial Networks (CMGAN) baseline model, which relaxes the existing constraints of precise phase and gives the neural network a broader learning space. Results show that this method achieves a new state-of-the-art performance without incurring additional computational overhead.
△ Less
Submitted 4 June, 2024; v1 submitted 13 February, 2024;
originally announced February 2024.
-
A computationally efficient semi-blind source separation based approach for nonlinear echo cancellation based on an element-wise iterative source steering
Authors:
Kunxing Lu,
Xianrui Wang,
Tetsuya Ueda,
Shoji Makino,
Jingdong Chen
Abstract:
While the semi-blind source separation-based acoustic echo cancellation (SBSS-AEC) has received much research attention due to its promising performance during double-talk compared to the traditional adaptive algorithms, it suffers from system latency and nonlinear distortions. To circumvent these drawbacks, the recently developed ideas on convolutive transfer function (CTF) approximation and nonl…
▽ More
While the semi-blind source separation-based acoustic echo cancellation (SBSS-AEC) has received much research attention due to its promising performance during double-talk compared to the traditional adaptive algorithms, it suffers from system latency and nonlinear distortions. To circumvent these drawbacks, the recently developed ideas on convolutive transfer function (CTF) approximation and nonlinear expansion have been used in the iterative projection (IP)-based semi-blind source separation (SBSS) algorithm. However, because of the introduction of CTF approximation and nonlinear expansion, this algorithm becomes computationally very expensive, which makes it difficult to implement in embedded systems. Thus, we attempt in this paper to improve this IP-based algorithm, thereby developing an element-wise iterative source steering (EISS) algorithm. In comparison with the IP-based SBSS algorithm, the proposed algorithm is computationally much more efficient, especially when the nonlinear expansion order is high and the length of the CTF filter is long. Meanwhile, its AEC performance is as good as that of IP-based SBSS.
△ Less
Submitted 13 December, 2023;
originally announced December 2023.
-
FastMVAE2: On improving and accelerating the fast variational autoencoder-based source separation algorithm for determined mixtures
Authors:
Li Li,
Hirokazu Kameoka,
Shoji Makino
Abstract:
This paper proposes a new source model and training scheme to improve the accuracy and speed of the multichannel variational autoencoder (MVAE) method. The MVAE method is a recently proposed powerful multichannel source separation method. It consists of pretraining a source model represented by a conditional VAE (CVAE) and then estimating separation matrices along with other unknown parameters so…
▽ More
This paper proposes a new source model and training scheme to improve the accuracy and speed of the multichannel variational autoencoder (MVAE) method. The MVAE method is a recently proposed powerful multichannel source separation method. It consists of pretraining a source model represented by a conditional VAE (CVAE) and then estimating separation matrices along with other unknown parameters so that the log-likelihood is non-decreasing given an observed mixture signal. Although the MVAE method has been shown to provide high source separation performance, one drawback is the computational cost of the backpropagation steps in the separation-matrix estimation algorithm. To overcome this drawback, a method called "FastMVAE" was subsequently proposed, which uses an auxiliary classifier VAE (ACVAE) to train the source model. By using the classifier and encoder trained in this way, the optimal parameters of the source model can be inferred efficiently, albeit approximately, in each step of the algorithm. However, the generalization capability of the trained ACVAE source model was not satisfactory, which led to poor performance in situations with unseen data. To improve the generalization capability, this paper proposes a new model architecture (called the "ChimeraACVAE" model) and a training scheme based on knowledge distillation. The experimental results revealed that the proposed source model trained with the proposed loss function achieved better source separation performance with less computation time than FastMVAE. We also confirmed that our methods were able to separate 18 sources with a reasonably good accuracy.
△ Less
Submitted 7 September, 2022; v1 submitted 28 September, 2021;
originally announced September 2021.
-
Speech Enhancement by Noise Self-Supervised Rank-Constrained Spatial Covariance Matrix Estimation via Independent Deeply Learned Matrix Analysis
Authors:
Sota Misawa,
Norihiro Takamune,
Tomohiko Nakamura,
Daichi Kitamura,
Hiroshi Saruwatari,
Masakazu Une,
Shoji Makino
Abstract:
Rank-constrained spatial covariance matrix estimation (RCSCME) is a method for the situation that the directional target speech and the diffuse noise are mixed. In conventional RCSCME, independent low-rank matrix analysis (ILRMA) is used as the preprocessing method. We propose RCSCME using independent deeply learned matrix analysis (IDLMA), which is a supervised extension of ILRMA. In this method,…
▽ More
Rank-constrained spatial covariance matrix estimation (RCSCME) is a method for the situation that the directional target speech and the diffuse noise are mixed. In conventional RCSCME, independent low-rank matrix analysis (ILRMA) is used as the preprocessing method. We propose RCSCME using independent deeply learned matrix analysis (IDLMA), which is a supervised extension of ILRMA. In this method, IDLMA requires deep neural networks (DNNs) to separate the target speech and the noise. We use Denoiser, which is a single-channel speech enhancement DNN, in IDLMA to estimate not only the target speech but also the noise. We also propose noise self-supervised RCSCME, in which we estimate the noise-only time intervals using the output of Denoiser and design the prior distribution of the noise spatial covariance matrix for RCSCME. We confirm that the proposed methods outperform the conventional methods under several noise conditions.
△ Less
Submitted 10 September, 2021;
originally announced September 2021.
-
Fast MVAE: Joint separation and classification of mixed sources based on multichannel variational autoencoder with auxiliary classifier
Authors:
Li Li,
Hirokazu Kameoka,
Shoji Makino
Abstract:
This paper proposes an alternative algorithm for multichannel variational autoencoder (MVAE), a recently proposed multichannel source separation approach. While MVAE is notable in its impressive source separation performance, the convergence-guaranteed optimization algorithm and that it allows us to estimate source-class labels simultaneously with source separation, there are still two major drawb…
▽ More
This paper proposes an alternative algorithm for multichannel variational autoencoder (MVAE), a recently proposed multichannel source separation approach. While MVAE is notable in its impressive source separation performance, the convergence-guaranteed optimization algorithm and that it allows us to estimate source-class labels simultaneously with source separation, there are still two major drawbacks, i.e., the high computational complexity and unsatisfactory source classification accuracy. To overcome these drawbacks, the proposed method employs an auxiliary classifier VAE, an information-theoretic extension of the conditional VAE, for learning the generative model of the source spectrograms. Furthermore, with the trained auxiliary classifier, we introduce a novel algorithm for the optimization that is able to not only reduce the computational time but also improve the source classification performance. We call the proposed method "fast MVAE (fMVAE)". Experimental evaluations revealed that fMVAE achieved comparative source separation performance to MVAE and about 80% source classification accuracy rate while it reduced about 93% computational time.
△ Less
Submitted 13 February, 2019; v1 submitted 16 December, 2018;
originally announced December 2018.
-
Semi-blind source separation with multichannel variational autoencoder
Authors:
Hirokazu Kameoka,
Li Li,
Shota Inoue,
Shoji Makino
Abstract:
This paper proposes a multichannel source separation technique called the multichannel variational autoencoder (MVAE) method, which uses a conditional VAE (CVAE) to model and estimate the power spectrograms of the sources in a mixture. By training the CVAE using the spectrograms of training examples with source-class labels, we can use the trained decoder distribution as a universal generative mod…
▽ More
This paper proposes a multichannel source separation technique called the multichannel variational autoencoder (MVAE) method, which uses a conditional VAE (CVAE) to model and estimate the power spectrograms of the sources in a mixture. By training the CVAE using the spectrograms of training examples with source-class labels, we can use the trained decoder distribution as a universal generative model capable of generating spectrograms conditioned on a specified class label. By treating the latent space variables and the class label as the unknown parameters of this generative model, we can develop a convergence-guaranteed semi-blind source separation algorithm that consists of iteratively estimating the power spectrograms of the underlying sources as well as the separation matrices. In experimental evaluations, our MVAE produced better separation performance than a baseline method.
△ Less
Submitted 26 August, 2018; v1 submitted 2 August, 2018;
originally announced August 2018.
-
Chromatic and High-frequency cVEP-based BCI Paradigm
Authors:
Daiki Aminaka,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
We present results of an approach to a code-modulated visual evoked potential (cVEP) based brain-computer interface (BCI) paradigm using four high-frequency flashing stimuli. To generate higher frequency stimulation compared to the state-of-the-art cVEP-based BCIs, we propose to use the light-emitting diodes (LEDs) driven from a small micro-controller board hardware generator designed by our team.…
▽ More
We present results of an approach to a code-modulated visual evoked potential (cVEP) based brain-computer interface (BCI) paradigm using four high-frequency flashing stimuli. To generate higher frequency stimulation compared to the state-of-the-art cVEP-based BCIs, we propose to use the light-emitting diodes (LEDs) driven from a small micro-controller board hardware generator designed by our team. The high-frequency and green-blue chromatic flashing stimuli are used in the study in order to minimize a danger of a photosensitive epilepsy (PSE). We compare the the green-blue chromatic cVEP-based BCI accuracies with the conventional white-black flicker based interface.
△ Less
Submitted 22 June, 2015; v1 submitted 14 June, 2015;
originally announced June 2015.
-
Inter-stimulus Interval Study for the Tactile Point-pressure Brain-computer Interface
Authors:
Kensuke Shimizu,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
The paper presents a study of an inter-stimulus interval (ISI) influence on a tactile point-pressure stimulus-based brain-computer interface's (tpBCI) classification accuracy. A novel tactile pressure generating tpBCI stimulator is also discussed, which is based on a three-by-three pins' matrix prototype. The six pin-linear patterns are presented to the user's palm during the online tpBCI experime…
▽ More
The paper presents a study of an inter-stimulus interval (ISI) influence on a tactile point-pressure stimulus-based brain-computer interface's (tpBCI) classification accuracy. A novel tactile pressure generating tpBCI stimulator is also discussed, which is based on a three-by-three pins' matrix prototype. The six pin-linear patterns are presented to the user's palm during the online tpBCI experiments in an oddball style paradigm allowing for "the aha-responses" elucidation, within the event related potential (ERP). A subsequent classification accuracies' comparison is discussed based on two ISI settings in an online tpBCI application. A research hypothesis of classification accuracies' non-significant differences with various ISIs is confirmed based on the two settings of 120 ms and 300 ms, as well as with various numbers of ERP response averaging scenarios.
△ Less
Submitted 14 June, 2015;
originally announced June 2015.
-
Head-related Impulse Response Cues for Spatial Auditory Brain-computer Interface
Authors:
Chisaki Nakaizumi,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
This study provides a comprehensive test of a head-related impulse response (HRIR) cues for a spatial auditory brain-computer interface (saBCI) speller paradigm. We present a comparison with the conventional virtual sound headphone-based spatial auditory modality. We propose and optimize the three types of sound spatialization settings using a variable elevation in order to evaluate the HRIR effic…
▽ More
This study provides a comprehensive test of a head-related impulse response (HRIR) cues for a spatial auditory brain-computer interface (saBCI) speller paradigm. We present a comparison with the conventional virtual sound headphone-based spatial auditory modality. We propose and optimize the three types of sound spatialization settings using a variable elevation in order to evaluate the HRIR efficacy for the saBCI. Three experienced and seven naive BCI users participated in the three experimental setups based on ten presented Japanese syllables. The obtained EEG auditory evoked potentials (AEP) resulted with encouragingly good and stable P300 responses in online BCI experiments. Our case study indicated that users could perceive elevation in the saBCI experiments generated using the HRIR measured from a general head model. The saBCI accuracy and information transfer rate (ITR) scores have been improved comparing to the classical horizontal plane-based virtual spatial sound reproduction modality, as far as the healthy users in the current pilot study are concerned.
△ Less
Submitted 14 June, 2015;
originally announced June 2015.
-
Spatial Tactile Brain-Computer Interface Paradigm Applying Vibration Stimuli to Large Areas of User's Back
Authors:
Takumi Kodama,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
We aim at an augmentation of communication abilities of amyotrophic lateral sclerosis (ALS) patients by creating a brain-computer interface (BCI) which can control a computer or other device by using only brain activity. As a method, we use a stimulus-driven BCI based on vibration stimuli delivered via a gaming pad to the user's back. We identify P300 responses from brain activity data in response…
▽ More
We aim at an augmentation of communication abilities of amyotrophic lateral sclerosis (ALS) patients by creating a brain-computer interface (BCI) which can control a computer or other device by using only brain activity. As a method, we use a stimulus-driven BCI based on vibration stimuli delivered via a gaming pad to the user's back. We identify P300 responses from brain activity data in response to the vibration stimuli. The user's intentions are classified according to the P300 responses recorded in the EEG. From the results of the psychophysical and online BCI experiments, we are able to classify the P300 responses very accurately, which proves the effectiveness of the proposed method.
△ Less
Submitted 16 April, 2014;
originally announced April 2014.
-
Head-related Impulse Response-based Spatial Auditory Brain-computer Interface
Authors:
Chisaki Nakaizumi,
Toshie Matsui,
Koichi Mori,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
This study provides a comprehensive test of the head-related impulse response (HRIR) to an auditory spatial speller brain-computer interface (BCI) paradigm, including a comparison with a conventional virtual headphone-based spatial auditory modality. Five BCI-naive users participated in an experiment based on five Japanese vowels. The auditory evoked potentials obtained produced encouragingly good…
▽ More
This study provides a comprehensive test of the head-related impulse response (HRIR) to an auditory spatial speller brain-computer interface (BCI) paradigm, including a comparison with a conventional virtual headphone-based spatial auditory modality. Five BCI-naive users participated in an experiment based on five Japanese vowels. The auditory evoked potentials obtained produced encouragingly good and stable P300-responses in online BCI experiments. Our case study indicates that the auditory HRIR spatial sound paradigm reproduced with headphones could be a viable alternative to established multi-loudspeaker surround sound BCI-speller applications.
△ Less
Submitted 15 April, 2014;
originally announced April 2014.
-
Auditory Brain-Computer Interface Paradigm with Head Related Impulse Response-based Spatial Cues
Authors:
Chisaki Nakaizumi,
Koichi Mori,
Toshie Matsui,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
The aim of this study is to provide a comprehensive test of head related impulse response (HRIR) for an auditory spatial speller brain-computer interface (BCI) paradigm. The study is conducted with six users in an experimental set up based on five Japanese hiragana vowels. Auditory evoked potentials resulted with encouragingly good and stable "aha-" or P300-responses in real-world online BCI exper…
▽ More
The aim of this study is to provide a comprehensive test of head related impulse response (HRIR) for an auditory spatial speller brain-computer interface (BCI) paradigm. The study is conducted with six users in an experimental set up based on five Japanese hiragana vowels. Auditory evoked potentials resulted with encouragingly good and stable "aha-" or P300-responses in real-world online BCI experiments. Our case study indicated that the auditory HRIR spatial sound reproduction paradigm could be a viable alternative to the established multi-loudspeaker surround sound BCI-speller applications, as far as healthy pilot study users are concerned.
△ Less
Submitted 14 December, 2013;
originally announced December 2013.
-
EEG Signal Processing and Classification for the Novel Tactile-Force Brain-Computer Interface Paradigm
Authors:
Shota Kono,
Daiki Aminaka,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
The presented study explores the extent to which tactile-force stimulus delivered to a hand holding a joystick can serve as a platform for a brain computer interface (BCI). The four pressure directions are used to evoke tactile brain potential responses, thus defining a tactile-force brain computer interface (tfBCI). We present brain signal processing and classification procedures leading to succe…
▽ More
The presented study explores the extent to which tactile-force stimulus delivered to a hand holding a joystick can serve as a platform for a brain computer interface (BCI). The four pressure directions are used to evoke tactile brain potential responses, thus defining a tactile-force brain computer interface (tfBCI). We present brain signal processing and classification procedures leading to successful interfacing results. Experimental results with seven subjects performing online BCI experiments provide a validation of the hand location tfBCI paradigm, while the feasibility of the concept is illuminated through remarkable information-transfer rates.
△ Less
Submitted 17 October, 2013; v1 submitted 6 October, 2013;
originally announced October 2013.
-
Multi-command Chest Tactile Brain Computer Interface for Small Vehicle Robot Navigation
Authors:
Hiromu Mori,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
The presented study explores the extent to which tactile stimuli delivered to five chest positions of a healthy user can serve as a platform for a brain computer interface (BCI) that could be used in an interactive application such as robotic vehicle operation. The five chest locations are used to evoke tactile brain potential responses, thus defining a tactile brain computer interface (tBCI). Exp…
▽ More
The presented study explores the extent to which tactile stimuli delivered to five chest positions of a healthy user can serve as a platform for a brain computer interface (BCI) that could be used in an interactive application such as robotic vehicle operation. The five chest locations are used to evoke tactile brain potential responses, thus defining a tactile brain computer interface (tBCI). Experimental results with five subjects performing online tBCI provide a validation of the chest location tBCI paradigm, while the feasibility of the concept is illuminated through information-transfer rates. Additionally an offline classification improvement with a linear SVM classifier is presented through the case study.
△ Less
Submitted 28 July, 2013;
originally announced July 2013.
-
Multi-command Tactile Brain Computer Interface: A Feasibility Study
Authors:
Hiromu Mori,
Yoshihiro Matsumoto,
Victor Kryssanov,
Eric Cooper,
Hitoshi Ogawa,
Shoji Makino,
Zbigniew R. Struzik,
Tomasz M. Rutkowski
Abstract:
The study presented explores the extent to which tactile stimuli delivered to the ten digits of a BCI-naive subject can serve as a platform for a brain computer interface (BCI) that could be used in an interactive application such as robotic vehicle operation. The ten fingertips are used to evoke somatosensory brain responses, thus defining a tactile brain computer interface (tBCI). Experimental r…
▽ More
The study presented explores the extent to which tactile stimuli delivered to the ten digits of a BCI-naive subject can serve as a platform for a brain computer interface (BCI) that could be used in an interactive application such as robotic vehicle operation. The ten fingertips are used to evoke somatosensory brain responses, thus defining a tactile brain computer interface (tBCI). Experimental results on subjects performing online (real-time) tBCI, using stimuli with a moderately fast inter-stimulus-interval (ISI), provide a validation of the tBCI prototype, while the feasibility of the concept is illuminated through information-transfer rates obtained through the case study.
△ Less
Submitted 18 May, 2013;
originally announced May 2013.
-
Comparison of P300 Responses in Auditory, Visual and Audiovisual Spatial Speller BCI Paradigms
Authors:
M. Chang,
N. Nishikawa,
Z. R. Struzik,
K. Mori,
S. Makino,
D. Mandic,
T. M. Rutkowski
Abstract:
The aim of this study is to provide a comprehensive test of three spatial speller settings, for the auditory, visual, and audiovisual paradigms. For rigour, the study is conducted with 16 BCI-naïve subjects in an experimental set-up based on five Japanese hiragana characters. Auditory P300 responses give encouragingly longer target vs. non-target latencies during the training phase, however, real-…
▽ More
The aim of this study is to provide a comprehensive test of three spatial speller settings, for the auditory, visual, and audiovisual paradigms. For rigour, the study is conducted with 16 BCI-naïve subjects in an experimental set-up based on five Japanese hiragana characters. Auditory P300 responses give encouragingly longer target vs. non-target latencies during the training phase, however, real-world online BCI experiments in the multimodal setting do not validate this potential advantage. Our case studies indicate that the auditory spatial unimodal paradigm needs further development in order to be a viable alternative to the established visual domain speller applications, as far as BCI-naïve subjects are concerned.
△ Less
Submitted 12 May, 2013; v1 submitted 27 January, 2013;
originally announced January 2013.
-
Multi-command Tactile and Auditory Brain Computer Interface based on Head Position Stimulation
Authors:
H. Mori,
Y. Matsumoto,
Z. R. Struzik,
K. Mori,
S. Makino,
D. Mandic,
T. M. Rutkowski
Abstract:
We study the extent to which vibrotactile stimuli delivered to the head of a subject can serve as a platform for a brain computer interface (BCI) paradigm. Six head positions are used to evoke combined somatosensory and auditory (via the bone conduction effect) brain responses, in order to define a multimodal tactile and auditory brain computer interface (taBCI). Experimental results of subjects p…
▽ More
We study the extent to which vibrotactile stimuli delivered to the head of a subject can serve as a platform for a brain computer interface (BCI) paradigm. Six head positions are used to evoke combined somatosensory and auditory (via the bone conduction effect) brain responses, in order to define a multimodal tactile and auditory brain computer interface (taBCI). Experimental results of subjects performing online taBCI, using stimuli with a moderately fast inter-stimulus interval (ISI), validate the taBCI paradigm, while the feasibility of the concept is illuminated through information transfer rate case studies.
△ Less
Submitted 12 May, 2013; v1 submitted 27 January, 2013;
originally announced January 2013.
-
Multicommand Tactile Brain Computer Interface based on Fingertips or Head Stimulation
Authors:
Hiromu Mori,
Yoshihiro Matsumoto,
Koichi Mori,
Victor Kryssanov,
Shoji Makino,
Zbigniew R. Struzik,
Gen Hori,
Tomasz M. Rutkowski
Abstract:
The paper presents results from a computational neuroscience study conducted to test vibrotactile stimuli delivered to subject fingertips and head areas in order to evoke the somatosensory brain responses utilized in a haptic brain computer interface (hBCI) paradigm. We present the preliminary and very encouraging results, with subjects conducting online hBCI interfacing experiments, ranging from…
▽ More
The paper presents results from a computational neuroscience study conducted to test vibrotactile stimuli delivered to subject fingertips and head areas in order to evoke the somatosensory brain responses utilized in a haptic brain computer interface (hBCI) paradigm. We present the preliminary and very encouraging results, with subjects conducting online hBCI interfacing experiments, ranging from 40% to 90% with a very fast inter-stimulus-interval (ISI) of 250ms. The presented results confirm our hypothesis that the hBCI paradigm concept is valid and it allows for rapid stimuli presentation in order to achieve a satisfactory information-transfer-rate of the novel BCI.
△ Less
Submitted 24 January, 2013; v1 submitted 11 November, 2012;
originally announced November 2012.
-
Psychophysical Responses Comparison in Spatial Visual, Audiovisual, and Auditory BCI-Spelling Paradigms
Authors:
Moonjeong Chang,
Nozomu Nishikawa,
Zhenyu Cai,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
The paper presents a pilot study conducted with spatial visual, audiovisual and auditory brain-computer-interface (BCI) based speller paradigms. The psychophysical experiments are conducted with healthy subjects in order to evaluate a difficulty and a possible response accuracy variability. We also present preliminary EEG results in offline BCI mode. The obtained results validate a thesis, that sp…
▽ More
The paper presents a pilot study conducted with spatial visual, audiovisual and auditory brain-computer-interface (BCI) based speller paradigms. The psychophysical experiments are conducted with healthy subjects in order to evaluate a difficulty and a possible response accuracy variability. We also present preliminary EEG results in offline BCI mode. The obtained results validate a thesis, that spatial auditory only paradigm performs as good as the traditional visual and audiovisual speller BCI tasks.
△ Less
Submitted 10 October, 2012;
originally announced October 2012.
-
The Spatial Real and Virtual Sound Stimuli Optimization for the Auditory BCI
Authors:
Nozomu Nishikawa,
Yoshihiro Matsumoto,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
The paper presents results from a project aiming to create horizontally distributed surround sound sources and virtual sound images as auditory BCI (aBCI) stimuli. The purpose is to create evoked brain wave response patterns depending on attended or ignored sound directions. We propose to use a modified version of the vector based amplitude panning (VBAP) approach to achieve the goal. The so creat…
▽ More
The paper presents results from a project aiming to create horizontally distributed surround sound sources and virtual sound images as auditory BCI (aBCI) stimuli. The purpose is to create evoked brain wave response patterns depending on attended or ignored sound directions. We propose to use a modified version of the vector based amplitude panning (VBAP) approach to achieve the goal. The so created spatial sound stimulus system for the novel oddball aBCI paradigm allows us to create a multi-command experimental environment with very encouraging results reported in this paper. We also present results showing that a modulation of the sound image depth changes also the subject responses. Finally, we also compare the proposed virtual sound approach with the traditional one based on real sound sources generated from the real loudspeaker directions. The so obtained results confirm the hypothesis of the possibility to modulate independently the brain responses to spatial types and depths of sound sources which allows for the development of the novel multi-command aBCI.
△ Less
Submitted 10 October, 2012;
originally announced October 2012.
-
Spatial Auditory BCI Paradigm Utilizing N200 and P300 Responses
Authors:
Zhenyu Cai,
Shoji Makino,
Takeshi Yamada,
Tomasz M. Rutkowski
Abstract:
The paper presents our recent results obtained with a new auditory spatial localization based BCI paradigm in which the ERP shape differences at early latencies are employed to enhance the traditional P300 responses in an oddball experimental setting. The concept relies on the recent results in auditory neuroscience showing a possibility to differentiate early anterior contralateral responses to a…
▽ More
The paper presents our recent results obtained with a new auditory spatial localization based BCI paradigm in which the ERP shape differences at early latencies are employed to enhance the traditional P300 responses in an oddball experimental setting. The concept relies on the recent results in auditory neuroscience showing a possibility to differentiate early anterior contralateral responses to attended spatial sources. Contemporary stimuli-driven BCI paradigms benefit mostly from the P300 ERP latencies in so called "aha-response" settings. We show the further enhancement of the classification results in spatial auditory paradigms by incorporating the N200 latencies, which differentiate the brain responses to lateral, in relation to the subject head, sound locations in the auditory space. The results reveal that those early spatial auditory ERPs boost online classification results of the BCI application. The online BCI experiments with the multi-command BCI prototype support our research hypothesis with the higher classification results and the improved information-transfer-rates.
△ Less
Submitted 10 October, 2012;
originally announced October 2012.
-
Auditory Steady-State Response Stimuli based BCI Application - The Optimization of the Stimuli Types and Lengths
Authors:
Yoshihiro Matsumoto,
Nozomu Nishikawa,
Takeshi Yamada,
Shoji Makino,
Tomasz M. Rutkowski
Abstract:
We propose a method for an improvement of auditory BCI (aBCI) paradigm based on a combination of ASSR stimuli optimization by choosing the subjects' best responses to AM-, flutter-, AM/FM and click-envelope modulated sounds. As the ASSR response features we propose pairwise phase-locking-values calculated from the EEG and next classified using binary classifier to detect attended and ignored stimu…
▽ More
We propose a method for an improvement of auditory BCI (aBCI) paradigm based on a combination of ASSR stimuli optimization by choosing the subjects' best responses to AM-, flutter-, AM/FM and click-envelope modulated sounds. As the ASSR response features we propose pairwise phase-locking-values calculated from the EEG and next classified using binary classifier to detect attended and ignored stimuli. We also report on a possibility to use the stimuli as short as half a second, which is a step forward in ASSR based aBCI. The presented results are helpful for optimization of the aBCI stimuli for each subject.
△ Less
Submitted 10 October, 2012;
originally announced October 2012.
-
Vibrotactile Stimulus Frequency Optimization for the Haptic BCI Prototype
Authors:
Hiromu Mori,
Yoshihiro Matsumito,
Shoji Makino,
Victor Kryssanov,
Tomasz M. Rutkowski
Abstract:
The paper presents results from a psychophysical study conducted to optimize vibrotactile stimuli delivered to subject finger tips in order to evoke the somatosensory responses to be utilized next in a haptic brain computer interface (hBCI) paradigm. We also present the preliminary EEG evoked responses for the chosen stimulating frequency. The obtained results confirm our hypothesis that the hBCI…
▽ More
The paper presents results from a psychophysical study conducted to optimize vibrotactile stimuli delivered to subject finger tips in order to evoke the somatosensory responses to be utilized next in a haptic brain computer interface (hBCI) paradigm. We also present the preliminary EEG evoked responses for the chosen stimulating frequency. The obtained results confirm our hypothesis that the hBCI paradigm concept is valid and it will allow for rapid stimuli presentation in order to improve information-transfer-rate (ITR) of the BCI.
△ Less
Submitted 10 October, 2012; v1 submitted 10 October, 2012;
originally announced October 2012.
-
Haptic BCI Paradigm based on Somatosensory Evoked Potential
Authors:
Tomasz M. Rutkowski,
Hiromu Mori,
Yoshihiro Matsumoto,
Zhenyu Cai,
Moonjeong Chang,
Nozomu Nishikawa,
Shoji Makino,
Koichi Mori
Abstract:
A new concept and an online prototype of haptic BCI paradigm are presented. Our main goal is to develop a new, alternative and low cost paradigm, with open-source hardware and software components. We also report results obtained with the novel dry EEG electrodes based signal acquisition system by g.tec, which further improves experimental comfort. We address the following points: a novel applicati…
▽ More
A new concept and an online prototype of haptic BCI paradigm are presented. Our main goal is to develop a new, alternative and low cost paradigm, with open-source hardware and software components. We also report results obtained with the novel dry EEG electrodes based signal acquisition system by g.tec, which further improves experimental comfort. We address the following points: a novel application of the BCI; a new methodological approach used compared to earlier projects; a new benefit for potential users of a BCI; the approach working online/in real-time; development of a novel stimuli delivery hardware and software. The results with five healthy subjects and discussion of future developments conclude this submission.
△ Less
Submitted 10 October, 2012; v1 submitted 24 July, 2012;
originally announced July 2012.
