Showing 1–32 of 32 results for author: Leng, Z

PVTransformer: Point-to-Voxel Transformer for Scalable 3D Object Detection

Authors: Zhaoqi Leng, Pei Sun, Tong He, Dragomir Anguelov, Mingxing Tan

Abstract: 3D object detectors for point clouds often rely on a pooling-based PointNet to encode sparse points into grid-like voxels or pillars. In this paper, we identify that the common PointNet design introduces an information bottleneck that limits 3D object detection accuracy and scalability. To address this limitation, we propose PVTransformer: a transformer-based point-to-voxel architecture for 3D det… ▽ More 3D object detectors for point clouds often rely on a pooling-based PointNet to encode sparse points into grid-like voxels or pillars. In this paper, we identify that the common PointNet design introduces an information bottleneck that limits 3D object detection accuracy and scalability. To address this limitation, we propose PVTransformer: a transformer-based point-to-voxel architecture for 3D detection. Our key idea is to replace the PointNet pooling operation with an attention module, leading to a better point-to-voxel aggregation function. Our design respects the permutation invariance of sparse 3D points while being more expressive than the pooling-based PointNet. Experimental results show our PVTransformer achieves much better performance compared to the latest 3D object detectors. On the widely used Waymo Open Dataset, our PVTransformer achieves state-of-the-art 76.5 mAPH L2, outperforming the prior art of SWFormer by +1.7 mAPH L2. △ Less

Submitted 5 May, 2024; originally announced May 2024.

HyperSDFusion: Bridging Hierarchical Structures in Language and Geometry for Enhanced 3D Text2Shape Generation

Authors: Zhiying Leng, Tolga Birdal, Xiaohui Liang, Federico Tombari

Abstract: 3D shape generation from text is a fundamental task in 3D representation learning. The text-shape pairs exhibit a hierarchical structure, where a general text like ``chair" covers all 3D shapes of the chair, while more detailed prompts refer to more specific shapes. Furthermore, both text and 3D shapes are inherently hierarchical structures. However, existing Text2Shape methods, such as SDFusion,… ▽ More 3D shape generation from text is a fundamental task in 3D representation learning. The text-shape pairs exhibit a hierarchical structure, where a general text like ``chair" covers all 3D shapes of the chair, while more detailed prompts refer to more specific shapes. Furthermore, both text and 3D shapes are inherently hierarchical structures. However, existing Text2Shape methods, such as SDFusion, do not exploit that. In this work, we propose HyperSDFusion, a dual-branch diffusion model that generates 3D shapes from a given text. Since hyperbolic space is suitable for handling hierarchical data, we propose to learn the hierarchical representations of text and 3D shapes in hyperbolic space. First, we introduce a hyperbolic text-image encoder to learn the sequential and multi-modal hierarchical features of text in hyperbolic space. In addition, we design a hyperbolic text-graph convolution module to learn the hierarchical features of text in hyperbolic space. In order to fully utilize these text features, we introduce a dual-branch structure to embed text features in 3D feature space. At last, to endow the generated 3D shapes with a hierarchical structure, we devise a hyperbolic hierarchical loss. Our method is the first to explore the hyperbolic hierarchical representation for text-to-shape generation. Experimental results on the existing text-to-shape paired dataset, Text2Shape, achieved state-of-the-art results. We release our implementation under HyperSDFusion.github.io. △ Less

Submitted 30 April, 2024; v1 submitted 1 March, 2024; originally announced March 2024.

Journal ref: IEEE/CVF conference on computer vision and pattern recognition 2024

IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition

Authors: Zikang Leng, Amitrajit Bhattacharjee, Hrudhai Rajasekhar, Lizhe Zhang, Elizabeth Bruda, Hyeokhyen Kwon, Thomas Plötz

Abstract: One of the primary challenges in the field of human activity recognition (HAR) is the lack of large labeled datasets. This hinders the development of robust and generalizable models. Recently, cross modality transfer approaches have been explored that can alleviate the problem of data scarcity. These approaches convert existing datasets from a source modality, such as video, to a target modality (… ▽ More One of the primary challenges in the field of human activity recognition (HAR) is the lack of large labeled datasets. This hinders the development of robust and generalizable models. Recently, cross modality transfer approaches have been explored that can alleviate the problem of data scarcity. These approaches convert existing datasets from a source modality, such as video, to a target modality (IMU). With the emergence of generative AI models such as large language models (LLMs) and text-driven motion synthesis models, language has become a promising source data modality as well as shown in proof of concepts such as IMUGPT. In this work, we conduct a large-scale evaluation of language-based cross modality transfer to determine their effectiveness for HAR. Based on this study, we introduce two new extensions for IMUGPT that enhance its use for practical HAR application scenarios: a motion filter capable of filtering out irrelevant motion sequences to ensure the relevance of the generated virtual IMU data, and a set of metrics that measure the diversity of the generated data facilitating the determination of when to stop generating virtual IMU data for both effective and efficient processing. We demonstrate that our diversity metrics can reduce the effort needed for the generation of virtual IMU data by at least 50%, which open up IMUGPT for practical use cases beyond a mere proof of concept. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Real-Time Asphalt Pavement Layer Thickness Prediction Using Ground-Penetrating Radar Based on a Modified Extended Common Mid-Point (XCMP) Approach

Authors: Siqi Wang, Zhen Leng, Xin Sui, Weiguang Zhang, Tao Ma, Zehui Zhu

Abstract: The conventional surface reflection method has been widely used to measure the asphalt pavement layer dielectric constant using ground-penetrating radar (GPR). This method may be inaccurate for in-service pavement thickness estimation with dielectric constant variation through the depth, which could be addressed using the extended common mid-point method (XCMP) with air-coupled GPR antennas. Howev… ▽ More The conventional surface reflection method has been widely used to measure the asphalt pavement layer dielectric constant using ground-penetrating radar (GPR). This method may be inaccurate for in-service pavement thickness estimation with dielectric constant variation through the depth, which could be addressed using the extended common mid-point method (XCMP) with air-coupled GPR antennas. However, the factors affecting the XCMP method on thickness prediction accuracy haven't been studied. Manual acquisition of key factors is required, which hinders its real-time applications. This study investigates the affecting factors and develops a modified XCMP method to allow automatic thickness prediction of in-service asphalt pavement with non-uniform dielectric properties through depth. A sensitivity analysis was performed, necessitating the accurate estimation of time of flights (TOFs) from antenna pairs. A modified XCMP method based on edge detection was proposed to allow real-time TOFs estimation, then dielectric constant and thickness predictions. Field tests using a multi-channel GPR system were performed for validation. Both the surface reflection and XCMP setups were conducted. Results show that the modified XCMP method is recommended with a mean prediction error of 1.86%, which is more accurate than the surface reflection method (5.73%). △ Less

Submitted 6 January, 2024; originally announced January 2024.

Comments: IEEE Transactions on Intelligent Transportation Systems (2024)

Hyper-Restormer: A General Hyperspectral Image Restoration Transformer for Remote Sensing Imaging

Authors: Yo-Yu Lai, Chia-Hsiang Lin, Zi-Chao Leng

Abstract: The deep learning model Transformer has achieved remarkable success in the hyperspectral image (HSI) restoration tasks by leveraging Spectral and Spatial Self-Attention (SA) mechanisms. However, applying these designs to remote sensing (RS) HSI restoration tasks, which involve far more spectrums than typical HSI (e.g., ICVL dataset with 31 bands), presents challenges due to the enormous computatio… ▽ More The deep learning model Transformer has achieved remarkable success in the hyperspectral image (HSI) restoration tasks by leveraging Spectral and Spatial Self-Attention (SA) mechanisms. However, applying these designs to remote sensing (RS) HSI restoration tasks, which involve far more spectrums than typical HSI (e.g., ICVL dataset with 31 bands), presents challenges due to the enormous computational complexity of using Spectral and Spatial SA mechanisms. To address this problem, we proposed Hyper-Restormer, a lightweight and effective Transformer-based architecture for RS HSI restoration. First, we introduce a novel Lightweight Spectral-Spatial (LSS) Transformer Block that utilizes both Spectral and Spatial SA to capture long-range dependencies of input features map. Additionally, we employ a novel Lightweight Locally-enhanced Feed-Forward Network (LLFF) to further enhance local context information. Then, LSS Transformer Blocks construct a Single-stage Lightweight Spectral-Spatial Transformer (SLSST) that cleverly utilizes the low-rank property of RS HSI to decompose the feature maps into basis and abundance components, enabling Spectral and Spatial SA with low computational cost. Finally, the proposed Hyper-Restormer cascades several SLSSTs in a stepwise manner to progressively enhance the quality of RS HSI restoration from coarse to fine. Extensive experiments were conducted on various RS HSI restoration tasks, including denoising, inpainting, and super-resolution, demonstrating that the proposed Hyper-Restormer outperforms other state-of-the-art methods. △ Less

Submitted 12 December, 2023; originally announced December 2023.

D-SCo: Dual-Stream Conditional Diffusion for Monocular Hand-Held Object Reconstruction

Authors: Bowen Fu, Gu Wang, Chenyangguang Zhang, Yan Di, Ziqin Huang, Zhiying Leng, Fabian Manhardt, Xiangyang Ji, Federico Tombari

Abstract: Reconstructing hand-held objects from a single RGB image is a challenging task in computer vision. In contrast to prior works that utilize deterministic modeling paradigms, we employ a point cloud denoising diffusion model to account for the probabilistic nature of this problem. In the core, we introduce centroid-fixed dual-stream conditional diffusion for monocular hand-held object reconstruction… ▽ More Reconstructing hand-held objects from a single RGB image is a challenging task in computer vision. In contrast to prior works that utilize deterministic modeling paradigms, we employ a point cloud denoising diffusion model to account for the probabilistic nature of this problem. In the core, we introduce centroid-fixed dual-stream conditional diffusion for monocular hand-held object reconstruction (D-SCo), tackling two predominant challenges. First, to avoid the object centroid from deviating, we utilize a novel hand-constrained centroid fixing paradigm, enhancing the stability of diffusion and reverse processes and the precision of feature projection. Second, we introduce a dual-stream denoiser to semantically and geometrically model hand-object interactions with a novel unified hand-object semantic embedding, enhancing the reconstruction performance of the hand-occluded region of the object. Experiments on the synthetic ObMan dataset and three real-world datasets HO3D, MOW and DexYCB demonstrate that our approach can surpass all other state-of-the-art methods. △ Less

Submitted 5 September, 2024; v1 submitted 23 November, 2023; originally announced November 2023.

Comments: ECCV 2024

InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews

Authors: Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, Yanghua Xiao

Abstract: Role-playing agents (RPAs), powered by large language models, have emerged as a flourishing field of applications. However, a key challenge lies in assessing whether RPAs accurately reproduce the personas of target characters, namely their character fidelity. Existing methods mainly focus on the knowledge and linguistic patterns of characters. This paper, instead, introduces a novel perspective to… ▽ More Role-playing agents (RPAs), powered by large language models, have emerged as a flourishing field of applications. However, a key challenge lies in assessing whether RPAs accurately reproduce the personas of target characters, namely their character fidelity. Existing methods mainly focus on the knowledge and linguistic patterns of characters. This paper, instead, introduces a novel perspective to evaluate the personality fidelity of RPAs with psychological scales. Overcoming drawbacks of previous self-report assessments on RPAs, we propose InCharacter, namely Interviewing Character agents for personality tests. Experiments include various types of RPAs and LLMs, covering 32 distinct characters on 14 widely used psychological scales. The results validate the effectiveness of InCharacter in measuring RPA personalities. Then, with InCharacter, we show that state-of-the-art RPAs exhibit personalities highly aligned with the human-perceived personalities of the characters, achieving an accuracy up to 80.7%. △ Less

Submitted 7 June, 2024; v1 submitted 27 October, 2023; originally announced October 2023.

Comments: ACL 2024

On the Benefit of Generative Foundation Models for Human Activity Recognition

Authors: Zikang Leng, Hyeokhyen Kwon, Thomas Plötz

Abstract: In human activity recognition (HAR), the limited availability of annotated data presents a significant challenge. Drawing inspiration from the latest advancements in generative AI, including Large Language Models (LLMs) and motion synthesis models, we believe that generative AI can address this data scarcity by autonomously generating virtual IMU data from text descriptions. Beyond this, we spotli… ▽ More In human activity recognition (HAR), the limited availability of annotated data presents a significant challenge. Drawing inspiration from the latest advancements in generative AI, including Large Language Models (LLMs) and motion synthesis models, we believe that generative AI can address this data scarcity by autonomously generating virtual IMU data from text descriptions. Beyond this, we spotlight several promising research pathways that could benefit from generative AI for the community, including the generating benchmark datasets, the development of foundational models specific to HAR, the exploration of hierarchical structures within HAR, breaking down complex activities, and applications in health sensing and activity summarization. △ Less

Submitted 18 October, 2023; originally announced October 2023.

Comments: Generative AI for Pervasive Computing (GenAI4PC) Symposium within UbiComp/ISWC 2023

LEF: Late-to-Early Temporal Fusion for LiDAR 3D Object Detection

Authors: Tong He, Pei Sun, Zhaoqi Leng, Chenxi Liu, Dragomir Anguelov, Mingxing Tan

Abstract: We propose a late-to-early recurrent feature fusion scheme for 3D object detection using temporal LiDAR point clouds. Our main motivation is fusing object-aware latent embeddings into the early stages of a 3D object detector. This feature fusion strategy enables the model to better capture the shapes and poses for challenging objects, compared with learning from raw points directly. Our method con… ▽ More We propose a late-to-early recurrent feature fusion scheme for 3D object detection using temporal LiDAR point clouds. Our main motivation is fusing object-aware latent embeddings into the early stages of a 3D object detector. This feature fusion strategy enables the model to better capture the shapes and poses for challenging objects, compared with learning from raw points directly. Our method conducts late-to-early feature fusion in a recurrent manner. This is achieved by enforcing window-based attention blocks upon temporally calibrated and aligned sparse pillar tokens. Leveraging bird's eye view foreground pillar segmentation, we reduce the number of sparse history features that our model needs to fuse into its current frame by 10$\times$. We also propose a stochastic-length FrameDrop training technique, which generalizes the model to variable frame lengths at inference for improved performance without retraining. We evaluate our method on the widely adopted Waymo Open Dataset and demonstrate improvement on 3D object detection against the baseline model, especially for the challenging category of large objects. △ Less

Submitted 28 September, 2023; originally announced September 2023.

Fg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Model

Authors: Yin Wang, Zhiying Leng, Frederick W. B. Li, Shun-Cheng Wu, Xiaohui Liang

Abstract: Text-driven human motion generation in computer vision is both significant and challenging. However, current methods are limited to producing either deterministic or imprecise motion sequences, failing to effectively control the temporal and spatial relationships required to conform to a given text description. In this work, we propose a fine-grained method for generating high-quality, conditional… ▽ More Text-driven human motion generation in computer vision is both significant and challenging. However, current methods are limited to producing either deterministic or imprecise motion sequences, failing to effectively control the temporal and spatial relationships required to conform to a given text description. In this work, we propose a fine-grained method for generating high-quality, conditional human motion sequences supporting precise text description. Our approach consists of two key components: 1) a linguistics-structure assisted module that constructs accurate and complete language feature to fully utilize text information; and 2) a context-aware progressive reasoning module that learns neighborhood and overall semantic linguistics features from shallow and deep graph neural networks to achieve a multi-step inference. Experiments show that our approach outperforms text-driven motion generation methods on HumanML3D and KIT test sets and generates better visually confirmed motion to the text conditions. △ Less

Submitted 12 September, 2023; originally announced September 2023.

Dynamic Hyperbolic Attention Network for Fine Hand-object Reconstruction

Authors: Zhiying Leng, Shun-Cheng Wu, Mahdi Saleh, Antonio Montanaro, Hao Yu, Yin Wang, Nassir Navab, Xiaohui Liang, Federico Tombari

Abstract: Reconstructing both objects and hands in 3D from a single RGB image is complex. Existing methods rely on manually defined hand-object constraints in Euclidean space, leading to suboptimal feature learning. Compared with Euclidean space, hyperbolic space better preserves the geometric properties of meshes thanks to its exponentially-growing space distance, which amplifies the differences between th… ▽ More Reconstructing both objects and hands in 3D from a single RGB image is complex. Existing methods rely on manually defined hand-object constraints in Euclidean space, leading to suboptimal feature learning. Compared with Euclidean space, hyperbolic space better preserves the geometric properties of meshes thanks to its exponentially-growing space distance, which amplifies the differences between the features based on similarity. In this work, we propose the first precise hand-object reconstruction method in hyperbolic space, namely Dynamic Hyperbolic Attention Network (DHANet), which leverages intrinsic properties of hyperbolic space to learn representative features. Our method that projects mesh and image features into a unified hyperbolic space includes two modules, ie. dynamic hyperbolic graph convolution and image-attention hyperbolic graph convolution. With these two modules, our method learns mesh features with rich geometry-image multi-modal information and models better hand-object interaction. Our method provides a promising alternative for fine hand-object reconstruction in hyperbolic space. Extensive experiments on three public datasets demonstrate that our method outperforms most state-of-the-art methods. △ Less

Submitted 6 September, 2023; originally announced September 2023.

Comments: Accpeted by ICCV 2023

ACM Class: I.4.5

ChatHaruhi: Reviving Anime Character in Reality via Large Language Model

Authors: Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi MI, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, Haozhen Sun

Abstract: Role-playing chatbots built on large language models have drawn interest, but better techniques are needed to enable mimicking specific fictional characters. We propose an algorithm that controls language models via an improved prompt and memories of the character extracted from scripts. We construct ChatHaruhi, a dataset covering 32 Chinese / English TV / anime characters with over 54k simulated… ▽ More Role-playing chatbots built on large language models have drawn interest, but better techniques are needed to enable mimicking specific fictional characters. We propose an algorithm that controls language models via an improved prompt and memories of the character extracted from scripts. We construct ChatHaruhi, a dataset covering 32 Chinese / English TV / anime characters with over 54k simulated dialogues. Both automatic and human evaluations show our approach improves role-playing ability over baselines. Code and data are available at https://github.com/LC1332/Chat-Haruhi-Suzumiya . △ Less

Submitted 18 August, 2023; originally announced August 2023.

Comments: v1 - First version of techique report

Generating Virtual On-body Accelerometer Data from Virtual Textual Descriptions for Human Activity Recognition

Authors: Zikang Leng, Hyeokhyen Kwon, Thomas Plötz

Abstract: The development of robust, generalized models in human activity recognition (HAR) has been hindered by the scarcity of large-scale, labeled data sets. Recent work has shown that virtual IMU data extracted from videos using computer vision techniques can lead to substantial performance improvements when training HAR models combined with small portions of real IMU data. Inspired by recent advances i… ▽ More The development of robust, generalized models in human activity recognition (HAR) has been hindered by the scarcity of large-scale, labeled data sets. Recent work has shown that virtual IMU data extracted from videos using computer vision techniques can lead to substantial performance improvements when training HAR models combined with small portions of real IMU data. Inspired by recent advances in motion synthesis from textual descriptions and connecting Large Language Models (LLMs) to various AI models, we introduce an automated pipeline that first uses ChatGPT to generate diverse textual descriptions of activities. These textual descriptions are then used to generate 3D human motion sequences via a motion synthesis model, T2M-GPT, and later converted to streams of virtual IMU data. We benchmarked our approach on three HAR datasets (RealWorld, PAMAP2, and USC-HAD) and demonstrate that the use of virtual IMU training data generated using our new approach leads to significantly improved HAR model performance compared to only using real IMU data. Our approach contributes to the growing field of cross-modality transfer methods and illustrate how HAR models can be improved through the generation of virtual training data that do not require any manual effort. △ Less

Submitted 4 May, 2023; originally announced May 2023.

WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecasting

Authors: Kan Chen, Runzhou Ge, Hang Qiu, Rami AI-Rfou, Charles R. Qi, Xuanyu Zhou, Zoey Yang, Scott Ettinger, Pei Sun, Zhaoqi Leng, Mustafa Baniodeh, Ivan Bogun, Weiyue Wang, Mingxing Tan, Dragomir Anguelov

Abstract: Widely adopted motion forecasting datasets substitute the observed sensory inputs with higher-level abstractions such as 3D boxes and polylines. These sparse shapes are inferred through annotating the original scenes with perception systems' predictions. Such intermediate representations tie the quality of the motion forecasting models to the performance of computer vision models. Moreover, the hu… ▽ More Widely adopted motion forecasting datasets substitute the observed sensory inputs with higher-level abstractions such as 3D boxes and polylines. These sparse shapes are inferred through annotating the original scenes with perception systems' predictions. Such intermediate representations tie the quality of the motion forecasting models to the performance of computer vision models. Moreover, the human-designed explicit interfaces between perception and motion forecasting typically pass only a subset of the semantic information present in the original sensory input. To study the effect of these modular approaches, design new paradigms that mitigate these limitations, and accelerate the development of end-to-end motion forecasting models, we augment the Waymo Open Motion Dataset (WOMD) with large-scale, high-quality, diverse LiDAR data for the motion forecasting task. The new augmented dataset WOMD-LiDAR consists of over 100,000 scenes that each spans 20 seconds, consisting of well-synchronized and calibrated high quality LiDAR point clouds captured across a range of urban and suburban geographies (https://waymo.com/open/data/motion/). Compared to Waymo Open Dataset (WOD), WOMD-LiDAR dataset contains 100x more scenes. Furthermore, we integrate the LiDAR data into the motion forecasting model training and provide a strong baseline. Experiments show that the LiDAR data brings improvement in the motion forecasting task. We hope that WOMD-LiDAR will provide new opportunities for boosting end-to-end motion forecasting models. △ Less

Submitted 18 February, 2024; v1 submitted 7 April, 2023; originally announced April 2023.

Comments: ICRA 2024 camera ready version. Dataset website: https://waymo.com/open/data/motion/

Fine-grained Human Activity Recognition Using Virtual On-body Acceleration Data

Authors: Zikang Leng, Yash Jain, Hyeokhyen Kwon, Thomas Plötz

Abstract: Previous work has demonstrated that virtual accelerometry data, extracted from videos using cross-modality transfer approaches like IMUTube, is beneficial for training complex and effective human activity recognition (HAR) models. Systems like IMUTube were originally designed to cover activities that are based on substantial body (part) movements. Yet, life is complex, and a range of activities of… ▽ More Previous work has demonstrated that virtual accelerometry data, extracted from videos using cross-modality transfer approaches like IMUTube, is beneficial for training complex and effective human activity recognition (HAR) models. Systems like IMUTube were originally designed to cover activities that are based on substantial body (part) movements. Yet, life is complex, and a range of activities of daily living is based on only rather subtle movements, which bears the question to what extent systems like IMUTube are of value also for fine-grained HAR, i.e., When does IMUTube break? In this work we first introduce a measure to quantitatively assess the subtlety of human movements that are underlying activities of interest--the motion subtlety index (MSI)--which captures local pixel movements and pose changes in the vicinity of target virtual sensor locations, and correlate it to the eventual activity recognition accuracy. We then perform a "stress-test" on IMUTube and explore for which activities with underlying subtle movements a cross-modality transfer approach works, and for which not. As such, the work presented in this paper allows us to map out the landscape for IMUTube applications in practical scenarios. △ Less

Submitted 2 November, 2022; originally announced November 2022.

LidarAugment: Searching for Scalable 3D LiDAR Data Augmentations

Authors: Zhaoqi Leng, Guowang Li, Chenxi Liu, Ekin Dogus Cubuk, Pei Sun, Tong He, Dragomir Anguelov, Mingxing Tan

Abstract: Data augmentations are important in training high-performance 3D object detectors for point clouds. Despite recent efforts on designing new data augmentations, perhaps surprisingly, most state-of-the-art 3D detectors only use a few simple data augmentations. In particular, different from 2D image data augmentations, 3D data augmentations need to account for different representations of input data… ▽ More Data augmentations are important in training high-performance 3D object detectors for point clouds. Despite recent efforts on designing new data augmentations, perhaps surprisingly, most state-of-the-art 3D detectors only use a few simple data augmentations. In particular, different from 2D image data augmentations, 3D data augmentations need to account for different representations of input data and require being customized for different models, which introduces significant overhead. In this paper, we resort to a search-based approach, and propose LidarAugment, a practical and effective data augmentation strategy for 3D object detection. Unlike previous approaches where all augmentation policies are tuned in an exponentially large search space, we propose to factorize and align the search space of each data augmentation, which cuts down the 20+ hyperparameters to 2, and significantly reduces the search complexity. We show LidarAugment can be customized for different model architectures with different input representations by a simple 2D grid search, and consistently improve both convolution-based UPillars/StarNet/RSN and transformer-based SWFormer. Furthermore, LidarAugment mitigates overfitting and allows us to scale up 3D detectors to much larger capacity. In particular, by combining with latest 3D detectors, our LidarAugment achieves a new state-of-the-art 74.8 mAPH L2 on Waymo Open Dataset. △ Less

Submitted 24 October, 2022; originally announced October 2022.

PseudoAugment: Learning to Use Unlabeled Data for Data Augmentation in Point Clouds

Authors: Zhaoqi Leng, Shuyang Cheng, Benjamin Caine, Weiyue Wang, Xiao Zhang, Jonathon Shlens, Mingxing Tan, Dragomir Anguelov

Abstract: Data augmentation is an important technique to improve data efficiency and save labeling cost for 3D detection in point clouds. Yet, existing augmentation policies have so far been designed to only utilize labeled data, which limits the data diversity. In this paper, we recognize that pseudo labeling and data augmentation are complementary, thus propose to leverage unlabeled data for data augmenta… ▽ More Data augmentation is an important technique to improve data efficiency and save labeling cost for 3D detection in point clouds. Yet, existing augmentation policies have so far been designed to only utilize labeled data, which limits the data diversity. In this paper, we recognize that pseudo labeling and data augmentation are complementary, thus propose to leverage unlabeled data for data augmentation to enrich the training data. In particular, we design three novel pseudo-label based data augmentation policies (PseudoAugments) to fuse both labeled and pseudo-labeled scenes, including frames (PseudoFrame), objecta (PseudoBBox), and background (PseudoBackground). PseudoAugments outperforms pseudo labeling by mitigating pseudo labeling errors and generating diverse fused training scenes. We demonstrate PseudoAugments generalize across point-based and voxel-based architectures, different model capacity and both KITTI and Waymo Open Dataset. To alleviate the cost of hyperparameter tuning and iterative pseudo labeling, we develop a population-based data augmentation framework for 3D detection, named AutoPseudoAugment. Unlike previous works that perform pseudo-labeling offline, our framework performs PseudoAugments and hyperparameter tuning in one shot to reduce computational cost. Experimental results on the large-scale Waymo Open Dataset show our method outperforms state-of-the-art auto data augmentation method (PPBA) and self-training method (pseudo labeling). In particular, AutoPseudoAugment is about 3X and 2X data efficient on vehicle and pedestrian tasks compared to prior arts. Notably, AutoPseudoAugment nearly matches the full dataset training results, with just 10% of the labeled run segments on the vehicle detection task. △ Less

Submitted 24 October, 2022; originally announced October 2022.

Journal ref: ECCV 2022 (pp. 555-572). Springer, Cham

SWFormer: Sparse Window Transformer for 3D Object Detection in Point Clouds

Authors: Pei Sun, Mingxing Tan, Weiyue Wang, Chenxi Liu, Fei Xia, Zhaoqi Leng, Dragomir Anguelov

Abstract: 3D object detection in point clouds is a core component for modern robotics and autonomous driving systems. A key challenge in 3D object detection comes from the inherent sparse nature of point occupancy within the 3D scene. In this paper, we propose Sparse Window Transformer (SWFormer ), a scalable and accurate model for 3D object detection, which can take full advantage of the sparsity of point… ▽ More 3D object detection in point clouds is a core component for modern robotics and autonomous driving systems. A key challenge in 3D object detection comes from the inherent sparse nature of point occupancy within the 3D scene. In this paper, we propose Sparse Window Transformer (SWFormer ), a scalable and accurate model for 3D object detection, which can take full advantage of the sparsity of point clouds. Built upon the idea of window-based Transformers, SWFormer converts 3D points into sparse voxels and windows, and then processes these variable-length sparse windows efficiently using a bucketing scheme. In addition to self-attention within each spatial window, our SWFormer also captures cross-window correlation with multi-scale feature fusion and window shifting operations. To further address the unique challenge of detecting 3D objects accurately from sparse features, we propose a new voxel diffusion technique. Experimental results on the Waymo Open Dataset show our SWFormer achieves state-of-the-art 73.36 L2 mAPH on vehicle and pedestrian for 3D object detection on the official test set, outperforming all previous single-stage and two-stage models, while being much more efficient. △ Less

Submitted 13 October, 2022; originally announced October 2022.

Journal ref: ECCV 2022

LidarNAS: Unifying and Searching Neural Architectures for 3D Point Clouds

Authors: Chenxi Liu, Zhaoqi Leng, Pei Sun, Shuyang Cheng, Charles R. Qi, Yin Zhou, Mingxing Tan, Dragomir Anguelov

Abstract: Developing neural models that accurately understand objects in 3D point clouds is essential for the success of robotics and autonomous driving. However, arguably due to the higher-dimensional nature of the data (as compared to images), existing neural architectures exhibit a large variety in their designs, including but not limited to the views considered, the format of the neural features, and th… ▽ More Developing neural models that accurately understand objects in 3D point clouds is essential for the success of robotics and autonomous driving. However, arguably due to the higher-dimensional nature of the data (as compared to images), existing neural architectures exhibit a large variety in their designs, including but not limited to the views considered, the format of the neural features, and the neural operations used. Lack of a unified framework and interpretation makes it hard to put these designs in perspective, as well as systematically explore new ones. In this paper, we begin by proposing a unified framework of such, with the key idea being factorizing the neural networks into a series of view transforms and neural layers. We demonstrate that this modular framework can reproduce a variety of existing works while allowing a fair comparison of backbone designs. Then, we show how this framework can easily materialize into a concrete neural architecture search (NAS) space, allowing a principled NAS-for-3D exploration. In performing evolutionary NAS on the 3D object detection task on the Waymo Open Dataset, not only do we outperform the state-of-the-art models, but also report the interesting finding that NAS tends to discover the same macro-level architecture concept for both the vehicle and pedestrian classes. △ Less

Submitted 10 October, 2022; originally announced October 2022.

Comments: ECCV 2022

The Effect of Sideslip on Jackknife Limits During Low Speed Trailer Operation

Authors: Zhe Leng, Mark A. Minor

Abstract: Jackknifing refers to the serious situation where a vehicle-trailer system enters a jackknife state and the vehicle and trailer eventually collide if trailer operation is not corrected. This paper considers low speed trailer maneuvering typical of trailer backing where jackknife state limits can vary due to sideslip caused by physical interaction between the vehicle, trailer, and environment. Anal… ▽ More Jackknifing refers to the serious situation where a vehicle-trailer system enters a jackknife state and the vehicle and trailer eventually collide if trailer operation is not corrected. This paper considers low speed trailer maneuvering typical of trailer backing where jackknife state limits can vary due to sideslip caused by physical interaction between the vehicle, trailer, and environment. Analysis of a kinematic model considering sideslip at the vehicle and trailer wheels indicates that vehicle-trailer systems should be divided into three categories based on the ratio of hitch length and trailer tongue length, each with distinct behaviors. The Long Trailer category may have no jackknifing state while the other two categories always have states leading to jackknifing. It is found that jackknife limits, which are the boundaries between the jackknifing state and the recoverable regions, can be divided into safe and unsafe limits, the latter of which must be avoided. Simulations and physical experiments support these results and provide insight about the implications of vehicle and trailer states with slip that lead to jackknifing. Simulations also demonstrate the benefit of considering these new slip-based jackknife limits in trailer backing control. △ Less

Submitted 14 July, 2022; originally announced July 2022.

Multi-Class 3D Object Detection with Single-Class Supervision

Authors: Mao Ye, Chenxi Liu, Maoqing Yao, Weiyue Wang, Zhaoqi Leng, Charles R. Qi, Dragomir Anguelov

Abstract: While multi-class 3D detectors are needed in many robotics applications, training them with fully labeled datasets can be expensive in labeling cost. An alternative approach is to have targeted single-class labels on disjoint data samples. In this paper, we are interested in training a multi-class 3D object detection model, while using these single-class labeled data. We begin by detailing the uni… ▽ More While multi-class 3D detectors are needed in many robotics applications, training them with fully labeled datasets can be expensive in labeling cost. An alternative approach is to have targeted single-class labels on disjoint data samples. In this paper, we are interested in training a multi-class 3D object detection model, while using these single-class labeled data. We begin by detailing the unique stance of our "Single-Class Supervision" (SCS) setting with respect to related concepts such as partial supervision and semi supervision. Then, based on the case study of training the multi-class version of Range Sparse Net (RSN), we adapt a spectrum of algorithms -- from supervised learning to pseudo-labeling -- to fully exploit the properties of our SCS setting, and perform extensive ablation studies to identify the most effective algorithm and practice. Empirical experiments on the Waymo Open Dataset show that proper training under SCS can approach or match full supervision training while saving labeling costs. △ Less

Submitted 11 May, 2022; originally announced May 2022.

Comments: ICRA 2022

PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functions

Authors: Zhaoqi Leng, Mingxing Tan, Chenxi Liu, Ekin Dogus Cubuk, Xiaojie Shi, Shuyang Cheng, Dragomir Anguelov

Abstract: Cross-entropy loss and focal loss are the most common choices when training deep neural networks for classification problems. Generally speaking, however, a good loss function can take on much more flexible forms, and should be tailored for different tasks and datasets. Motivated by how functions can be approximated via Taylor expansion, we propose a simple framework, named PolyLoss, to view and d… ▽ More Cross-entropy loss and focal loss are the most common choices when training deep neural networks for classification problems. Generally speaking, however, a good loss function can take on much more flexible forms, and should be tailored for different tasks and datasets. Motivated by how functions can be approximated via Taylor expansion, we propose a simple framework, named PolyLoss, to view and design loss functions as a linear combination of polynomial functions. Our PolyLoss allows the importance of different polynomial bases to be easily adjusted depending on the targeting tasks and datasets, while naturally subsuming the aforementioned cross-entropy loss and focal loss as special cases. Extensive experimental results show that the optimal choice within the PolyLoss is indeed dependent on the task and dataset. Simply by introducing one extra hyperparameter and adding one line of code, our Poly-1 formulation outperforms the cross-entropy loss and focal loss on 2D image classification, instance segmentation, object detection, and 3D object detection tasks, sometimes by a large margin. △ Less

Submitted 10 May, 2022; v1 submitted 26 April, 2022; originally announced April 2022.

Comments: Add ablation studies on COCO detection using RetinaNet (Section 8)

Journal ref: International Conference on Learning Representations. 2021

Improving 3D Object Detection through Progressive Population Based Augmentation

Authors: Shuyang Cheng, Zhaoqi Leng, Ekin Dogus Cubuk, Barret Zoph, Chunyan Bai, Jiquan Ngiam, Yang Song, Benjamin Caine, Vijay Vasudevan, Congcong Li, Quoc V. Le, Jonathon Shlens, Dragomir Anguelov

Abstract: Data augmentation has been widely adopted for object detection in 3D point clouds. However, all previous related efforts have focused on manually designing specific data augmentation methods for individual architectures. In this work, we present the first attempt to automate the design of data augmentation policies for 3D object detection. We introduce the Progressive Population Based Augmentation… ▽ More Data augmentation has been widely adopted for object detection in 3D point clouds. However, all previous related efforts have focused on manually designing specific data augmentation methods for individual architectures. In this work, we present the first attempt to automate the design of data augmentation policies for 3D object detection. We introduce the Progressive Population Based Augmentation (PPBA) algorithm, which learns to optimize augmentation strategies by narrowing down the search space and adopting the best parameters discovered in previous iterations. On the KITTI 3D detection test set, PPBA improves the StarNet detector by substantial margins on the moderate difficulty category of cars, pedestrians, and cyclists, outperforming all current state-of-the-art single-stage detection models. Additional experiments on the Waymo Open Dataset indicate that PPBA continues to effectively improve the StarNet and PointPillars detectors on a 20x larger dataset compared to KITTI. The magnitude of the improvements may be comparable to advances in 3D perception architectures and the gains come without an incurred cost at inference time. In subsequent experiments, we find that PPBA may be up to 10x more data efficient than baseline 3D detection models without augmentation, highlighting that 3D detection models may achieve competitive accuracy with far fewer labeled examples. △ Less

Submitted 16 July, 2020; v1 submitted 2 April, 2020; originally announced April 2020.

Comments: Accepted at ECCV 2020

Learning to mirror speaking styles incrementally

Authors: Siyi Liu, Ziang Leng, Derry Wijaya

Abstract: Mirroring is the behavior in which one person subconsciously imitates the gesture, speech pattern, or attitude of another. In conversations, mirroring often signals the speakers enjoyment and engagement in their communication. In chatbots, methods have been proposed to add personas to the chatbots and to train them to speak or to shift their dialogue style to that of the personas. However, they of… ▽ More Mirroring is the behavior in which one person subconsciously imitates the gesture, speech pattern, or attitude of another. In conversations, mirroring often signals the speakers enjoyment and engagement in their communication. In chatbots, methods have been proposed to add personas to the chatbots and to train them to speak or to shift their dialogue style to that of the personas. However, they often require a large dataset consisting of dialogues of the target personalities to train. In this work, we explore a method that can learn to mirror the speaking styles of a person incrementally. Our method extracts ngrams that capture a persons speaking styles and uses the ngrams to create patterns for transforming sentences to the persons speaking styles. Our experiments show that our method is able to capture patterns of speaking style that can be used to transform regular sentences into sentences with the target style. △ Less

Submitted 4 March, 2020; originally announced March 2020.

Comments: 4 pages, 3 tables, 1 figure

New material platform for superconducting transmon qubits with coherence times exceeding 0.3 milliseconds

Authors: Alex P. M. Place, Lila V. H. Rodgers, Pranav Mundada, Basil M. Smitham, Mattias Fitzpatrick, Zhaoqi Leng, Anjali Premkumar, Jacob Bryon, Sara Sussman, Guangming Cheng, Trisha Madhavan, Harshvardhan K. Babla, Berthold Jaeck, Andras Gyenis, Nan Yao, Robert J. Cava, Nathalie P. de Leon, Andrew A. Houck

Abstract: The superconducting transmon qubit is a leading platform for quantum computing and quantum science. Building large, useful quantum systems based on transmon qubits will require significant improvements in qubit relaxation and coherence times, which are orders of magnitude shorter than limits imposed by bulk properties of the constituent materials. This indicates that relaxation likely originates f… ▽ More The superconducting transmon qubit is a leading platform for quantum computing and quantum science. Building large, useful quantum systems based on transmon qubits will require significant improvements in qubit relaxation and coherence times, which are orders of magnitude shorter than limits imposed by bulk properties of the constituent materials. This indicates that relaxation likely originates from uncontrolled surfaces, interfaces, and contaminants. Previous efforts to improve qubit lifetimes have focused primarily on designs that minimize contributions from surfaces. However, significant improvements in the lifetime of two-dimensional transmon qubits have remained elusive for several years. Here, we fabricate two-dimensional transmon qubits that have both lifetimes and coherence times with dynamical decoupling exceeding 0.3 milliseconds by replacing niobium with tantalum in the device. We have observed increased lifetimes for seventeen devices, indicating that these material improvements are robust, paving the way for higher gate fidelities in multi-qubit processors. △ Less

Submitted 28 February, 2020; originally announced March 2020.

Noncollinear Spintronics and Electric-Field Control: A Review

Authors: Peixin Qin, Han Yan, Xiaoning Wang, Zexin Feng, Huixin Guo, Xiaorong Zhou, Haojiang Wu, Xin Zhang, Zhaoguogang Leng, Hongyu Chen, Zhiqi Liu

Abstract: Our world is composed of various materials with different structures, where spin structures have been playing a pivotal role in spintronic devices of the contemporary information technology. Apart from conventional collinear spin materials such as collinear ferromagnets and collinear antiferromagnetically coupled materials, noncollinear spintronic materials have emerged as hot spots of research at… ▽ More Our world is composed of various materials with different structures, where spin structures have been playing a pivotal role in spintronic devices of the contemporary information technology. Apart from conventional collinear spin materials such as collinear ferromagnets and collinear antiferromagnetically coupled materials, noncollinear spintronic materials have emerged as hot spots of research attention owing to exotic physical phenomena. In this Review, we firstly introduce two types noncollinear spin structures, i.e., the chiral spin structure that yields real-space Berry phases and the coplanar noncollinear spin structure that could generate momentum-space Berry phases, and then move to relevant novel physical phenomena including topological Hall effect, anomalous Hall effect, multiferroic, Weyl fermions, spin-polarized current, and spin Hall effect without spin-orbit coupling in these noncollinear spin systems. Afterwards, we summarize and elaborate the electric-field control of the noncollinear spin structure and related physical effects, which could enable ultralow power spintronic devices in future. In the final outlook part, we emphasize the importance and possible routes for experimentally detecting the intriguing theoretically predicted spin-polarized current, verifying the spin Hall effect in the absence of spin-orbit coupling and exploring the anisotropic magnetoresistance and domain-wall-related magnetoresistance effects for noncollinear antiferromagnetic materials. △ Less

Submitted 1 January, 2020; originally announced January 2020.

Comments: 34 pages, 20 figures, Review Article in Rare Metals

Integration of the Noncollinear Antiferromagnetic Metal Mn3Sn onto Ferroelectric Oxides for Electric-Field Control

Authors: Xiaoning Wang, Zexin Feng, Peixin Qin, Han Yan, Xiaorong Zhou, Huixin Guo, Zhaoguogang Leng, Weiqi Chen, Qiannan Jia, Zexiang Hu, Haojiang Wu, Xin Zhang, Chengbao Jiang, Zhiqi Liu

Abstract: Non-collinear antiferromagnetic materials have received dramatically increasing attention in the field of spintronics as their exotic topological features such as the Berry-curvature-induced anomalous Hall effect and possible magnetic Weyl states could be utilized in future topological antiferromagnetic spintronic devices. In this work, we report the successful integration of the antiferromagnetic… ▽ More Non-collinear antiferromagnetic materials have received dramatically increasing attention in the field of spintronics as their exotic topological features such as the Berry-curvature-induced anomalous Hall effect and possible magnetic Weyl states could be utilized in future topological antiferromagnetic spintronic devices. In this work, we report the successful integration of the antiferromagnetic metal Mn3Sn thin films onto ferroelectric oxide PMN-PT. By optimizing growth, we realized the large anomalous Hall effect with small switching magnetic fields of several tens mT fully comparable to those of bulk Mn3Sn single crystals, anisotropic magnetoresistance and negative parallel magnetoresistance in Mn3Sn thin films with antiferromagnetic order, which are similar to the signatures of the Weyl state in bulk Mn3Sn single crystals. More importantly, we found that the anomalous Hall effect in antiferromagnetic Mn3Sn thin films can be manipulated by electric fields applied onto the ferroelectric materials, thus demonstrating the feasibility of Mn3Sn-based topological spintronic devices operated in an ultralow power manner. △ Less

Submitted 15 October, 2019; originally announced October 2019.

Comments: 25 pages, 10 figure. Accepted in Acta Materialia

Journal ref: Acta Materialia 181, 537 (2019)

Robust and efficient algorithms for high-dimensional black-box quantum optimization

Authors: Zhaoqi Leng, Pranav Mundada, Saeed Ghadimi, Andrew Houck

Abstract: Hybrid quantum-classical optimization using near-term quantum technology is an emerging direction for exploring quantum advantage in high-dimensional systems. However, precise characterization of all experimental parameters is often impractical and challenging. A viable approach is to use algorithms that rely only on black-box inference rather than analytical gradients. Here, we combine randomized… ▽ More Hybrid quantum-classical optimization using near-term quantum technology is an emerging direction for exploring quantum advantage in high-dimensional systems. However, precise characterization of all experimental parameters is often impractical and challenging. A viable approach is to use algorithms that rely only on black-box inference rather than analytical gradients. Here, we combine randomized perturbation gradient estimation with adaptive momentum gradient updates to create the AdamSPSA and AdamRSGF algorithms. We prove the asymptotic convergence of our algorithms in a convex setting, and we benchmark them against other gradient-based optimization algorithms on non-convex optimal control tasks. Our results show that these new algorithms accelerate the convergence rate, decrease the variance of loss trajectories, and efficiently tune up high-fidelity (above 99.9\%) Hann-window single-qubit gates from trivial initial conditions with twenty variables. △ Less

Submitted 10 October, 2019; v1 submitted 8 October, 2019; originally announced October 2019.

Deep Learning-Enhanced Variational Monte Carlo Method for Quantum Many-Body Physics

Authors: Li Yang, Zhaoqi Leng, Guangyuan Yu, Ankit Patel, Wen-Jun Hu, Han Pu

Abstract: Artificial neural networks have been successfully incorporated into variational Monte Carlo method (VMC) to study quantum many-body systems. However, there have been few systematic studies of exploring quantum many-body physics using deep neural networks (DNNs), despite of the tremendous success enjoyed by DNNs in many other areas in recent years. One main challenge of implementing DNN in VMC is t… ▽ More Artificial neural networks have been successfully incorporated into variational Monte Carlo method (VMC) to study quantum many-body systems. However, there have been few systematic studies of exploring quantum many-body physics using deep neural networks (DNNs), despite of the tremendous success enjoyed by DNNs in many other areas in recent years. One main challenge of implementing DNN in VMC is the inefficiency of optimizing such networks with large number of parameters. We introduce an importance sampling gradient optimization (ISGO) algorithm, which significantly improves the computational speed of training DNN in VMC. We design an efficient convolutional DNN architecture to compute the ground state of a one-dimensional (1D) SU($N$) spin chain. Our numerical results of the ground-state energies with up to 16 layers of DNN show excellent agreement with the Bethe-Ansatz exact solution. Furthermore, we also calculate the loop correlation function using the wave function obtained. Our work demonstrates the feasibility and advantages of applying DNNs to numerical quantum many-body calculations. △ Less

Submitted 21 August, 2019; v1 submitted 26 May, 2019; originally announced May 2019.

Comments: 8 pages, 7 figures

Journal ref: Phys. Rev. Research 2, 012039 (2020)

A Piezoelectric, Strain-Controlled Antiferromagnetic Memory Insensitive to Magnetic Fields

Authors: Han Yan, Zexin Feng, Shunli Shang, Xiaoning Wang, Zexiang Hu, Jinhua Wang, Zengwei Zhu, Hui Wang, Zuhuang Chen, Hui Hua, Wenkuo Lu, Jingmin Wang, Peixin Qin, Huixin Guo, Xiaorong Zhou, Zhaoguogang Leng, Zikui Liu, Chengbao Jiang, Michael Coey, Zhiqi Liu

Abstract: Spintronic devices based on antiferromagnetic (AFM) materials hold the promise of fast switching speeds and robustness against magnetic fields. Different device concepts have been predicted and experimentally demonstrated, such as low-temperature AFM tunnel junctions that operate as spin-valves, or room-temperature AFM memory, for which either thermal heating in combination with magnetic fields, o… ▽ More Spintronic devices based on antiferromagnetic (AFM) materials hold the promise of fast switching speeds and robustness against magnetic fields. Different device concepts have been predicted and experimentally demonstrated, such as low-temperature AFM tunnel junctions that operate as spin-valves, or room-temperature AFM memory, for which either thermal heating in combination with magnetic fields, or Néel spin-orbit torque is used for the information writing process. On the other hand, piezoelectric materials were employed to control magnetism by electric fields in multiferroic heterostructures, which suppresses Joule heating caused by switching currents and may enable low energy-consuming electronic devices. Here, we combine the two material classes to explore changes of the resistance of the high-Néel-temperature antiferromagnet MnPt induced by piezoelectric strain. We find two non-volatile resistance states at room temperature and zero electric field, which are stable in magnetic fields up to 60 T. Furthermore, the strain-induced resistance switching process is insensitive to magnetic fields. Integration in a tunnel junction can further amplify the electroresistance. The tunneling anisotropic magnetoresistance reaches ~11.2% at room temperature. Overall, we demonstrate a piezoelectric, strain-controlled AFM memory which is fully operational in strong magnetic fields and has potential for low-energy and high-density memory applications. △ Less

Submitted 11 January, 2019; originally announced January 2019.

Comments: 9 pages

Journal ref: Nature Nanotechnology 14, 131-136 (2019)

Direct digital synthesis of microwave waveforms for quantum computing

Authors: J. Raftery, A. Vrajitoarea, G. Zhang, Z. Leng, S. J. Srinivasan, A. A. Houck

Abstract: Current state of the art quantum computing experiments in the microwave regime use control pulses generated by modulating microwave tones with baseband signals generated by an arbitrary waveform generator (AWG). Recent advances in digital analog conversion technology have made it possible to directly synthesize arbitrary microwave pulses with sampling rates up to 92 gigasamples per second (GS/s).… ▽ More Current state of the art quantum computing experiments in the microwave regime use control pulses generated by modulating microwave tones with baseband signals generated by an arbitrary waveform generator (AWG). Recent advances in digital analog conversion technology have made it possible to directly synthesize arbitrary microwave pulses with sampling rates up to 92 gigasamples per second (GS/s). These new high bandwidth AWG's could dramatically simplify the classical control chain for quantum computing experiments, enabling more advanced pulse shaping and reducing the number of components that need to be carefully calibrated. Here we use a high speed AWG to study the viability of such a simplified scheme. We characterize the AWG and perform randomized benchmarking of a superconducting qubit, achieving average single qubit gate error rates below $5\times10^{-4}$. △ Less

Submitted 2 March, 2017; originally announced March 2017.

Comments: 6 pages, 3 figures

Table-top Measurement of Local Magnetization Dynamics Using Picosecond Thermal Gradients: Toward Nanoscale Magnetic Imaging

Authors: J. M. Bartell, D. H. Ngai, Z. Leng, G. D. Fuchs

Abstract: Recent advances in nanoscale magnetism have demonstrated the potential for spin-based technology including magnetic random access memory, nanoscale microwave sources, and ultra-low power signal transfer. Future engineering advances and new scientific discoveries will be enabled by research tools capable of examining local magnetization dynamics at length and time scales fundamental to spatiotempor… ▽ More Recent advances in nanoscale magnetism have demonstrated the potential for spin-based technology including magnetic random access memory, nanoscale microwave sources, and ultra-low power signal transfer. Future engineering advances and new scientific discoveries will be enabled by research tools capable of examining local magnetization dynamics at length and time scales fundamental to spatiotemporal variations in magnetic systems - typically 10 - 200 nm and 5 - 50 ps. A key problem is that current table-top magnetic microscopy cannot access both of these scales simultaneously. In this letter, we introduce a spatiotemporal magnetic microscopy that uses magneto-thermoelectric interactions to measure local magnetization via the time-resolved anomalous Nernst effect (TRANE). By generating a short-lived, local thermal gradient, the magnetic moment is transduced into an electrical signal. Experimentally, we show that TRANE microscopy has time resolution below 30 ps and spatial resolution limited by the thermal excitation area. Furthermore, we present numerical simulations to show that the thermal spot size sets the limits of the spatial resolution, even at 50 nm. The thermal effects used for TRANE microscopy have no fundamental limit on their spatial resolution, therefore a future TRANE microscope employing a scanning plasmon antenna could enable measurements of nanoscale magnetic dynamics. △ Less

Submitted 23 February, 2015; originally announced February 2015.

Comments: 31 pages including supplementary information