Showing 1–50 of 66 results for author: Weng, T

Resolving the pressure induced 'self-insertion' in skutterudite CoSb3

Authors: Bihan Wang, Anna Pakhomova, Saiana Khandarkhaeva, Mirtha Pillaca, Peter Gille, Zhe Ren, Dmitry Lapkin, Dameli Assalauova, Pavel Alexeev, Ilya Sergeev, Satishkumar Kulkarni, Tsu-Chien Weng, Michael Sprung, Hanns-Peter Liermann, Ivan A. Vartanyants, Konstantin Glazyrin

Abstract: CoSb3, a skutterudite compound, is key in studying thermoelectric materials. Under compression, it undergoes a 'self-insertion' isostructural transition, redistributing large Sb atoms among crystallographic sites. We investigated CoSb3's structural stability up to 70 GPa using single crystal X-ray diffraction and high-resolution X-ray scattering, including Bragg Coherent Diffraction Imaging. We ex… ▽ More CoSb3, a skutterudite compound, is key in studying thermoelectric materials. Under compression, it undergoes a 'self-insertion' isostructural transition, redistributing large Sb atoms among crystallographic sites. We investigated CoSb3's structural stability up to 70 GPa using single crystal X-ray diffraction and high-resolution X-ray scattering, including Bragg Coherent Diffraction Imaging. We examined the material in three pressure transmitting media (PTMs), exploring how PTMs and nonhydrostatic stresses affect CoSb3. Notably, the 'self-insertion' transition may reduce or even make compressibility negative. Additionally, we report a previously unknown phase transformation from cubic Im-3 to trigonal R-3 above 40 GPa and discuss the phases' distinctive behaviors. △ Less

Submitted 27 August, 2024; originally announced August 2024.

Comments: Manuscript: 27 pages, 12 Figures

VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidance

Authors: Divyansh Srivastava, Ge Yan, Tsui-Wei Weng

Abstract: Concept Bottleneck Models (CBMs) provide interpretable prediction by introducing an intermediate Concept Bottleneck Layer (CBL), which encodes human-understandable concepts to explain models' decision. Recent works proposed to utilize Large Language Models (LLMs) and pre-trained Vision-Language Models (VLMs) to automate the training of CBMs, making it more scalable and automated. However, existing… ▽ More Concept Bottleneck Models (CBMs) provide interpretable prediction by introducing an intermediate Concept Bottleneck Layer (CBL), which encodes human-understandable concepts to explain models' decision. Recent works proposed to utilize Large Language Models (LLMs) and pre-trained Vision-Language Models (VLMs) to automate the training of CBMs, making it more scalable and automated. However, existing approaches still fall short in two aspects: First, the concepts predicted by CBL often mismatch the input image, raising doubts about the faithfulness of interpretation. Second, it has been shown that concept values encode unintended information: even a set of random concepts could achieve comparable test accuracy to state-of-the-art CBMs. To address these critical limitations, in this work, we propose a novel framework called Vision-Language-Guided Concept Bottleneck Model (VLG-CBM) to enable faithful interpretability with the benefits of boosted performance. Our method leverages off-the-shelf open-domain grounded object detectors to provide visually grounded concept annotation, which largely enhances the faithfulness of concept prediction while further improving the model performance. In addition, we propose a new metric called Number of Effective Concepts (NEC) to control the information leakage and provide better interpretability. Extensive evaluations across five standard benchmarks show that our method, VLG-CBM, outperforms existing methods by at least 4.27% and up to 51.09% on accuracy at NEC=5, and by at least 0.45% and up to 29.78% on average accuracy across different NECs, while preserves both faithfulness and interpretability of the learned concepts as demonstrated in extensive experiments. △ Less

Submitted 18 July, 2024; originally announced August 2024.

Crafting Large Language Models for Enhanced Interpretability

Authors: Chung-En Sun, Tuomas Oikarinen, Tsui-Wei Weng

Abstract: We introduce the Concept Bottleneck Large Language Model (CB-LLM), a pioneering approach to creating inherently interpretable Large Language Models (LLMs). Unlike traditional black-box LLMs that rely on post-hoc interpretation methods with limited neuron function insights, CB-LLM sets a new standard with its built-in interpretability, scalability, and ability to provide clear, accurate explanation… ▽ More We introduce the Concept Bottleneck Large Language Model (CB-LLM), a pioneering approach to creating inherently interpretable Large Language Models (LLMs). Unlike traditional black-box LLMs that rely on post-hoc interpretation methods with limited neuron function insights, CB-LLM sets a new standard with its built-in interpretability, scalability, and ability to provide clear, accurate explanations. This innovation not only advances transparency in language models but also enhances their effectiveness. Our unique Automatic Concept Correction (ACC) strategy successfully narrows the performance gap with conventional black-box LLMs, positioning CB-LLM as a model that combines the high accuracy of traditional LLMs with the added benefit of clear interpretability -- a feature markedly absent in existing LLMs. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Comments: Present at ICML 2024 Mechanistic Interpretability (MI) Workshop

Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agents

Authors: Chung-En Sun, Sicun Gao, Tsui-Wei Weng

Abstract: Robustness remains a paramount concern in deep reinforcement learning (DRL), with randomized smoothing emerging as a key technique for enhancing this attribute. However, a notable gap exists in the performance of current smoothed DRL agents, often characterized by significantly low clean rewards and weak robustness. In response to this challenge, our study introduces innovative algorithms aimed at… ▽ More Robustness remains a paramount concern in deep reinforcement learning (DRL), with randomized smoothing emerging as a key technique for enhancing this attribute. However, a notable gap exists in the performance of current smoothed DRL agents, often characterized by significantly low clean rewards and weak robustness. In response to this challenge, our study introduces innovative algorithms aimed at training effective smoothed robust DRL agents. We propose S-DQN and S-PPO, novel approaches that demonstrate remarkable improvements in clean rewards, empirical robustness, and robustness guarantee across standard RL benchmarks. Notably, our S-DQN and S-PPO agents not only significantly outperform existing smoothed agents by an average factor of $2.16\times$ under the strongest attack, but also surpass previous robustly-trained agents by an average factor of $2.13\times$. This represents a significant leap forward in the field. Furthermore, we introduce Smoothed Attack, which is $1.89\times$ more effective in decreasing the rewards of smoothed agents than existing adversarial attacks. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Comments: Published in ICML 2024

AND: Audio Network Dissection for Interpreting Deep Acoustic Models

Authors: Tung-Yu Wu, Yu-Xiang Lin, Tsui-Wei Weng

Abstract: Neuron-level interpretations aim to explain network behaviors and properties by investigating neurons responsive to specific perceptual or structural input patterns. Although there is emerging work in the vision and language domains, none is explored for acoustic models. To bridge the gap, we introduce $\textit{AND}$, the first $\textbf{A}$udio $\textbf{N}$etwork $\textbf{D}$issection framework th… ▽ More Neuron-level interpretations aim to explain network behaviors and properties by investigating neurons responsive to specific perceptual or structural input patterns. Although there is emerging work in the vision and language domains, none is explored for acoustic models. To bridge the gap, we introduce $\textit{AND}$, the first $\textbf{A}$udio $\textbf{N}$etwork $\textbf{D}$issection framework that automatically establishes natural language explanations of acoustic neurons based on highly-responsive audio. $\textit{AND}$ features the use of LLMs to summarize mutual acoustic features and identities among audio. Extensive experiments are conducted to verify $\textit{AND}$'s precise and informative descriptions. In addition, we demonstrate a potential use of $\textit{AND}$ for audio machine unlearning by conducting concept-specific pruning based on the generated descriptions. Finally, we highlight two acoustic model behaviors with analysis by $\textit{AND}$: (i) models discriminate audio with a combination of basic acoustic features rather than high-level abstract concepts; (ii) training strategies affect model behaviors and neuron interpretability -- supervised training guides neurons to gradually narrow their attention, while self-supervised learning encourages neurons to be polysemantic for exploring high-level features. △ Less

Submitted 26 June, 2024; v1 submitted 24 June, 2024; originally announced June 2024.

Comments: Accepted by ICML'24

Journal ref: Forty-first International Conference on Machine Learning (2024)

Quantum Computing in Wireless Communications and Networking: A Tutorial-cum-Survey

Authors: Wei Zhao, Tangjie Weng, Yue Ruan, Zhi Liu, Xuangou Wu, Xiao Zheng, Nei Kato

Abstract: Owing to its outstanding parallel computing capabilities, quantum computing (QC) has been a subject of continuous attention. With the gradual maturation of QC platforms, it has increasingly played a significant role in various fields such as transportation, pharmaceuticals, and industrial manufacturing,achieving unprecedented milestones. In modern society, wireless communication stands as an indis… ▽ More Owing to its outstanding parallel computing capabilities, quantum computing (QC) has been a subject of continuous attention. With the gradual maturation of QC platforms, it has increasingly played a significant role in various fields such as transportation, pharmaceuticals, and industrial manufacturing,achieving unprecedented milestones. In modern society, wireless communication stands as an indispensable infrastructure, with its essence lying in optimization. Although artificial intelligence (AI) algorithms such as Reinforcement Learning (RL) and mathematical optimization have greatly enhanced the performance of wireless communication, the rapid attainment of optimal solutions for wireless communication problems remains an unresolved challenge. QC, however, presents a new alternative. This paper aims to elucidate the fundamentals of QC and explore its applications in wireless communications and networking. First, we will provide a tutorial on QC, covering its basics, characteristics, and popular QC algorithms. Next, we will introduce the applications of QC in communication and networking, followed by its applications in miscellaneous areas such as security and privacy, localization and tracking, and video streaming. Finally,we will discuss remaining open issues before concluding. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Linear Explanations for Individual Neurons

Authors: Tuomas Oikarinen, Tsui-Wei Weng

Abstract: In recent years many methods have been developed to understand the internal workings of neural networks, often by describing the function of individual neurons in the model. However, these methods typically only focus on explaining the very highest activations of a neuron. In this paper we show this is not sufficient, and that the highest activation range is only responsible for a very small perce… ▽ More In recent years many methods have been developed to understand the internal workings of neural networks, often by describing the function of individual neurons in the model. However, these methods typically only focus on explaining the very highest activations of a neuron. In this paper we show this is not sufficient, and that the highest activation range is only responsible for a very small percentage of the neuron's causal effect. In addition, inputs causing lower activations are often very different and can't be reliably predicted by only looking at high activations. We propose that neurons should instead be understood as a linear combination of concepts, and develop an efficient method for producing these linear explanations. In addition, we show how to automatically evaluate description quality using simulation, i.e. predicting neuron activations on unseen inputs in vision setting. △ Less

Submitted 10 May, 2024; originally announced May 2024.

Comments: Published in ICML 2024

Provably Robust Conformal Prediction with Improved Efficiency

Authors: Ge Yan, Yaniv Romano, Tsui-Wei Weng

Abstract: Conformal prediction is a powerful tool to generate uncertainty sets with guaranteed coverage using any predictive model, under the assumption that the training and test data are i.i.d.. Recently, it has been shown that adversarial examples are able to manipulate conformal methods to construct prediction sets with invalid coverage rates, as the i.i.d. assumption is violated. To address this issue,… ▽ More Conformal prediction is a powerful tool to generate uncertainty sets with guaranteed coverage using any predictive model, under the assumption that the training and test data are i.i.d.. Recently, it has been shown that adversarial examples are able to manipulate conformal methods to construct prediction sets with invalid coverage rates, as the i.i.d. assumption is violated. To address this issue, a recent work, Randomized Smoothed Conformal Prediction (RSCP), was first proposed to certify the robustness of conformal prediction methods to adversarial noise. However, RSCP has two major limitations: (i) its robustness guarantee is flawed when used in practice and (ii) it tends to produce large uncertainty sets. To address these limitations, we first propose a novel framework called RSCP+ to provide provable robustness guarantee in evaluation, which fixes the issues in the original RSCP method. Next, we propose two novel methods, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), to effectively reduce prediction set size with little computation overhead. Experimental results in CIFAR10, CIFAR100, and ImageNet suggest the baseline method only yields trivial predictions including full label set, while our methods could boost the efficiency by up to $4.36\times$, $5.46\times$, and $16.9\times$ respectively and provide practical robustness guarantee. Our codes are available at https://github.com/Trustworthy-ML-Lab/Provably-Robust-Conformal-Prediction. △ Less

Submitted 30 April, 2024; originally announced April 2024.

Describe-and-Dissect: Interpreting Neurons in Vision Networks with Language Models

Authors: Nicholas Bai, Rahul A. Iyer, Tuomas Oikarinen, Tsui-Wei Weng

Abstract: In this paper, we propose Describe-and-Dissect (DnD), a novel method to describe the roles of hidden neurons in vision networks. DnD utilizes recent advancements in multimodal deep learning to produce complex natural language descriptions, without the need for labeled training data or a predefined set of concepts to choose from. Additionally, DnD is training-free, meaning we don't train any new mo… ▽ More In this paper, we propose Describe-and-Dissect (DnD), a novel method to describe the roles of hidden neurons in vision networks. DnD utilizes recent advancements in multimodal deep learning to produce complex natural language descriptions, without the need for labeled training data or a predefined set of concepts to choose from. Additionally, DnD is training-free, meaning we don't train any new models and can easily leverage more capable general purpose models in the future. We have conducted extensive qualitative and quantitative analysis to show that DnD outperforms prior work by providing higher quality neuron descriptions. Specifically, our method on average provides the highest quality labels and is more than 2 times as likely to be selected as the best explanation for a neuron than the best baseline. △ Less

Submitted 20 March, 2024; originally announced March 2024.

One step closer to unbiased aleatoric uncertainty estimation

Authors: Wang Zhang, Ziwen Ma, Subhro Das, Tsui-Wei Weng, Alexandre Megretski, Luca Daniel, Lam M. Nguyen

Abstract: Neural networks are powerful tools in various applications, and quantifying their uncertainty is crucial for reliable decision-making. In the deep learning field, the uncertainties are usually categorized into aleatoric (data) and epistemic (model) uncertainty. In this paper, we point out that the existing popular variance attenuation method highly overestimates aleatoric uncertainty. To address t… ▽ More Neural networks are powerful tools in various applications, and quantifying their uncertainty is crucial for reliable decision-making. In the deep learning field, the uncertainties are usually categorized into aleatoric (data) and epistemic (model) uncertainty. In this paper, we point out that the existing popular variance attenuation method highly overestimates aleatoric uncertainty. To address this issue, we propose a new estimation method by actively de-noising the observed data. By conducting a broad range of experiments, we demonstrate that our proposed approach provides a much closer approximation to the actual data uncertainty than the standard method. △ Less

Submitted 20 December, 2023; v1 submitted 16 December, 2023; originally announced December 2023.

Accurate Segmentation of Optic Disc And Cup from Multiple Pseudo-labels by Noise-aware Learning

Authors: Tengjin Weng, Yang Shen, Zhidong Zhao, Zhiming Cheng, Shuai Wang

Abstract: Optic disc and cup segmentation plays a crucial role in automating the screening and diagnosis of optic glaucoma. While data-driven convolutional neural networks (CNNs) show promise in this area, the inherent ambiguity of segmenting objects and background boundaries in the task of optic disc and cup segmentation leads to noisy annotations that impact model performance. To address this, we propose… ▽ More Optic disc and cup segmentation plays a crucial role in automating the screening and diagnosis of optic glaucoma. While data-driven convolutional neural networks (CNNs) show promise in this area, the inherent ambiguity of segmenting objects and background boundaries in the task of optic disc and cup segmentation leads to noisy annotations that impact model performance. To address this, we propose an innovative label-denoising method of Multiple Pseudo-labels Noise-aware Network (MPNN) for accurate optic disc and cup segmentation. Specifically, the Multiple Pseudo-labels Generation and Guided Denoising (MPGGD) module generates pseudo-labels by multiple different initialization networks trained on true labels, and the pixel-level consensus information extracted from these pseudo-labels guides to differentiate clean pixels from noisy pixels. The training framework of the MPNN is constructed by a teacher-student architecture to learn segmentation from clean pixels and noisy pixels. Particularly, such a framework adeptly leverages (i) reliable and fundamental insight from clean pixels and (ii) the supplementary knowledge within noisy pixels via multiple perturbation-based unsupervised consistency. Compared to other label-denoising methods, comprehensive experimental results on the RIGA dataset demonstrate our method's excellent performance. The code is available at https://github.com/wwwtttjjj/MPNN △ Less

Submitted 15 March, 2024; v1 submitted 30 November, 2023; originally announced November 2023.

Comments: CSCWD 2024

PMP-Swin: Multi-Scale Patch Message Passing Swin Transformer for Retinal Disease Classification

Authors: Zhihan Yang, Zhiming Cheng, Tengjin Weng, Shucheng He, Yaqi Wang, Xin Ye, Shuai Wang

Abstract: Retinal disease is one of the primary causes of visual impairment, and early diagnosis is essential for preventing further deterioration. Nowadays, many works have explored Transformers for diagnosing diseases due to their strong visual representation capabilities. However, retinal diseases exhibit milder forms and often present with overlapping signs, which pose great difficulties for accurate mu… ▽ More Retinal disease is one of the primary causes of visual impairment, and early diagnosis is essential for preventing further deterioration. Nowadays, many works have explored Transformers for diagnosing diseases due to their strong visual representation capabilities. However, retinal diseases exhibit milder forms and often present with overlapping signs, which pose great difficulties for accurate multi-class classification. Therefore, we propose a new framework named Multi-Scale Patch Message Passing Swin Transformer for multi-class retinal disease classification. Specifically, we design a Patch Message Passing (PMP) module based on the Message Passing mechanism to establish global interaction for pathological semantic features and to exploit the subtle differences further between different diseases. Moreover, considering the various scale of pathological features we integrate multiple PMP modules for different patch sizes. For evaluation, we have constructed a new dataset, named OPTOS dataset, consisting of 1,033 high-resolution fundus images photographed by Optos camera and conducted comprehensive experiments to validate the efficacy of our proposed method. And the results on both the public dataset and our dataset demonstrate that our method achieves remarkable performance compared to state-of-the-art methods. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Comments: 9 pages, 7 figures

MSE-Nets: Multi-annotated Semi-supervised Ensemble Networks for Improving Segmentation of Medical Image with Ambiguous Boundaries

Authors: Shuai Wang, Tengjin Weng, Jingyi Wang, Yang Shen, Zhidong Zhao, Yixiu Liu, Pengfei Jiao, Zhiming Cheng, Yaqi Wang

Abstract: Medical image segmentation annotations exhibit variations among experts due to the ambiguous boundaries of segmented objects and backgrounds in medical images. Although using multiple annotations for each image in the fully-supervised has been extensively studied for training deep models, obtaining a large amount of multi-annotated data is challenging due to the substantial time and manpower costs… ▽ More Medical image segmentation annotations exhibit variations among experts due to the ambiguous boundaries of segmented objects and backgrounds in medical images. Although using multiple annotations for each image in the fully-supervised has been extensively studied for training deep models, obtaining a large amount of multi-annotated data is challenging due to the substantial time and manpower costs required for segmentation annotations, resulting in most images lacking any annotations. To address this, we propose Multi-annotated Semi-supervised Ensemble Networks (MSE-Nets) for learning segmentation from limited multi-annotated and abundant unannotated data. Specifically, we introduce the Network Pairwise Consistency Enhancement (NPCE) module and Multi-Network Pseudo Supervised (MNPS) module to enhance MSE-Nets for the segmentation task by considering two major factors: (1) to optimize the utilization of all accessible multi-annotated data, the NPCE separates (dis)agreement annotations of multi-annotated data at the pixel level and handles agreement and disagreement annotations in different ways, (2) to mitigate the introduction of imprecise pseudo-labels, the MNPS extends the training data by leveraging consistent pseudo-labels from unannotated data. Finally, we improve confidence calibration by averaging the predictions of base networks. Experiments on the ISIC dataset show that we reduced the demand for multi-annotated data by 97.75\% and narrowed the gap with the best fully-supervised baseline to just a Jaccard index of 4\%. Furthermore, compared to other semi-supervised methods that rely only on a single annotation or a combined fusion approach, the comprehensive experimental results on ISIC and RIGA datasets demonstrate the superior performance of our proposed method in medical image segmentation with ambiguous boundaries. △ Less

Submitted 17 November, 2023; originally announced November 2023.

Corrupting Neuron Explanations of Deep Visual Features

Authors: Divyansh Srivastava, Tuomas Oikarinen, Tsui-Wei Weng

Abstract: The inability of DNNs to explain their black-box behavior has led to a recent surge of explainability methods. However, there are growing concerns that these explainability methods are not robust and trustworthy. In this work, we perform the first robustness analysis of Neuron Explanation Methods under a unified pipeline and show that these explanations can be significantly corrupted by random noi… ▽ More The inability of DNNs to explain their black-box behavior has led to a recent surge of explainability methods. However, there are growing concerns that these explainability methods are not robust and trustworthy. In this work, we perform the first robustness analysis of Neuron Explanation Methods under a unified pipeline and show that these explanations can be significantly corrupted by random noises and well-designed perturbations added to their probing data. We find that even adding small random noise with a standard deviation of 0.02 can already change the assigned concepts of up to 28% neurons in the deeper layers. Furthermore, we devise a novel corruption algorithm and show that our algorithm can manipulate the explanation of more than 80% neurons by poisoning less than 10% of probing data. This raises the concern of trusting Neuron Explanation Methods in real-life safety and fairness critical applications. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Journal ref: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 1877-1886

Promoting Robustness of Randomized Smoothing: Two Cost-Effective Approaches

Authors: Linbo Liu, Trong Nghia Hoang, Lam M. Nguyen, Tsui-Wei Weng

Abstract: Randomized smoothing has recently attracted attentions in the field of adversarial robustness to provide provable robustness guarantees on smoothed neural network classifiers. However, existing works show that vanilla randomized smoothing usually does not provide good robustness performance and often requires (re)training techniques on the base classifier in order to boost the robustness of the re… ▽ More Randomized smoothing has recently attracted attentions in the field of adversarial robustness to provide provable robustness guarantees on smoothed neural network classifiers. However, existing works show that vanilla randomized smoothing usually does not provide good robustness performance and often requires (re)training techniques on the base classifier in order to boost the robustness of the resulting smoothed classifier. In this work, we propose two cost-effective approaches to boost the robustness of randomized smoothing while preserving its clean performance. The first approach introduces a new robust training method AdvMacerwhich combines adversarial training and robustness certification maximization for randomized smoothing. We show that AdvMacer can improve the robustness performance of randomized smoothing classifiers compared to SOTA baselines, while being 3x faster to train than MACER baseline. The second approach introduces a post-processing method EsbRS which greatly improves the robustness certificate based on building model ensembles. We explore different aspects of model ensembles that has not been studied by prior works and propose a novel design methodology to further improve robustness of the ensemble based on our theoretical analysis. △ Less

Submitted 11 October, 2023; originally announced October 2023.

The Importance of Prompt Tuning for Automated Neuron Explanations

Authors: Justin Lee, Tuomas Oikarinen, Arjun Chatha, Keng-Chi Chang, Yilan Chen, Tsui-Wei Weng

Abstract: Recent advances have greatly increased the capabilities of large language models (LLMs), but our understanding of the models and their safety has not progressed as fast. In this paper we aim to understand LLMs deeper by studying their individual neurons. We build upon previous work showing large language models such as GPT-4 can be useful in explaining what each neuron in a language model does. Sp… ▽ More Recent advances have greatly increased the capabilities of large language models (LLMs), but our understanding of the models and their safety has not progressed as fast. In this paper we aim to understand LLMs deeper by studying their individual neurons. We build upon previous work showing large language models such as GPT-4 can be useful in explaining what each neuron in a language model does. Specifically, we analyze the effect of the prompt used to generate explanations and show that reformatting the explanation prompt in a more natural way can significantly improve neuron explanation quality and greatly reduce computational cost. We demonstrate the effects of our new prompts in three different ways, incorporating both automated and human evaluations. △ Less

Submitted 11 October, 2023; v1 submitted 9 October, 2023; originally announced October 2023.

Prediction without Preclusion: Recourse Verification with Reachable Sets

Authors: Avni Kothari, Bogdan Kulynych, Tsui-Wei Weng, Berk Ustun

Abstract: Machine learning models are often used to decide who receives a loan, a job interview, or a public benefit. Models in such settings use features without considering their actionability. As a result, they can assign predictions that are fixed $-$ meaning that individuals who are denied loans and interviews are, in fact, precluded from access to credit and employment. In this work, we introduce a pr… ▽ More Machine learning models are often used to decide who receives a loan, a job interview, or a public benefit. Models in such settings use features without considering their actionability. As a result, they can assign predictions that are fixed $-$ meaning that individuals who are denied loans and interviews are, in fact, precluded from access to credit and employment. In this work, we introduce a procedure called recourse verification to test if a model assigns fixed predictions to its decision subjects. We propose a model-agnostic approach for recourse verification with reachable sets $-$ i.e., the set of all points that a person can reach through their actions in feature space. We develop methods to construct reachable sets for discrete feature spaces, which can certify the responsiveness of any model by simply querying its predictions. We conduct a comprehensive empirical study on the infeasibility of recourse on datasets from consumer finance. Our results highlight how models can inadvertently preclude access by assigning fixed predictions and underscore the need to account for actionability in model development. △ Less

Submitted 1 May, 2024; v1 submitted 24 August, 2023; originally announced August 2023.

Comments: ICLR 2024 Spotlight. The first two authors contributed equally

Enhancing Point Annotations with Superpixel and Confidence Learning Guided for Improving Semi-Supervised OCT Fluid Segmentation

Authors: Tengjin Weng, Yang Shen, Kai Jin, Zhiming Cheng, Yunxiang Li, Gewen Zhang, Shuai Wang, Yaqi Wang

Abstract: Automatic segmentation of fluid in Optical Coherence Tomography (OCT) images is beneficial for ophthalmologists to make an accurate diagnosis. Although semi-supervised OCT fluid segmentation networks enhance their performance by introducing additional unlabeled data, the performance enhancement is limited. To address this, we propose Superpixel and Confident Learning Guide Point Annotations Networ… ▽ More Automatic segmentation of fluid in Optical Coherence Tomography (OCT) images is beneficial for ophthalmologists to make an accurate diagnosis. Although semi-supervised OCT fluid segmentation networks enhance their performance by introducing additional unlabeled data, the performance enhancement is limited. To address this, we propose Superpixel and Confident Learning Guide Point Annotations Network (SCLGPA-Net) based on the teacher-student architecture, which can learn OCT fluid segmentation from limited fully-annotated data and abundant point-annotated data. Specifically, we use points to annotate fluid regions in unlabeled OCT images and the Superpixel-Guided Pseudo-Label Generation (SGPLG) module generates pseudo-labels and pixel-level label trust maps from the point annotations. The label trust maps provide an indication of the reliability of the pseudo-labels. Furthermore, we propose the Confident Learning Guided Label Refinement (CLGLR) module identifies error information in the pseudo-labels and leads to further refinement. Experiments on the RETOUCH dataset show that we are able to reduce the need for fully-annotated data by 94.22\%, closing the gap with the best fully supervised baselines to a mean IoU of only 2\%. Furthermore, We constructed a private 2D OCT fluid segmentation dataset for evaluation. Compared with other methods, comprehensive experimental results demonstrate that the proposed method can achieve excellent performance in OCT fluid segmentation. △ Less

Submitted 30 November, 2023; v1 submitted 5 June, 2023; originally announced June 2023.

Comments: Submission to BSPC

Concept-Monitor: Understanding DNN training through individual neurons

Authors: Mohammad Ali Khan, Tuomas Oikarinen, Tsui-Wei Weng

Abstract: In this work, we propose a general framework called Concept-Monitor to help demystify the black-box DNN training processes automatically using a novel unified embedding space and concept diversity metric. Concept-Monitor enables human-interpretable visualization and indicators of the DNN training processes and facilitates transparency as well as deeper understanding on how DNNs develop along the d… ▽ More In this work, we propose a general framework called Concept-Monitor to help demystify the black-box DNN training processes automatically using a novel unified embedding space and concept diversity metric. Concept-Monitor enables human-interpretable visualization and indicators of the DNN training processes and facilitates transparency as well as deeper understanding on how DNNs develop along the during training. Inspired by these findings, we also propose a new training regularizer that incentivizes hidden neurons to learn diverse concepts, which we show to improve training performance. Finally, we apply Concept-Monitor to conduct several case studies on different training paradigms including adversarial training, fine-tuning and network pruning via the Lottery Ticket Hypothesis △ Less

Submitted 26 April, 2023; originally announced April 2023.

Learning imaging mechanism directly from optical microscopy observations

Authors: Ze-Hao Wang, Long-Kun Shan, Tong-Tian Weng, Tian-Long Chen, Qi-Yu Wang, Xiang-Dong Chen, Zhang-Yang Wang, Guang-Can Guo, Fang-Wen Sun

Abstract: Optical microscopy image plays an important role in scientific research through the direct visualization of the nanoworld, where the imaging mechanism is described as the convolution of the point spread function (PSF) and emitters. Based on a priori knowledge of the PSF or equivalent PSF, it is possible to achieve more precise exploration of the nanoworld. However, it is an outstanding challenge t… ▽ More Optical microscopy image plays an important role in scientific research through the direct visualization of the nanoworld, where the imaging mechanism is described as the convolution of the point spread function (PSF) and emitters. Based on a priori knowledge of the PSF or equivalent PSF, it is possible to achieve more precise exploration of the nanoworld. However, it is an outstanding challenge to directly extract the PSF from microscopy images. Here, with the help of self-supervised learning, we propose a physics-informed masked autoencoder (PiMAE) that enables a learnable estimation of the PSF and emitters directly from the raw microscopy images. We demonstrate our method in synthetic data and real-world experiments with significant accuracy and noise robustness. PiMAE outperforms DeepSTORM and the Richardson-Lucy algorithm in synthetic data tasks with an average improvement of 19.6\% and 50.7\% (35 tasks), respectively, as measured by the normalized root mean square error (NRMSE) metric. This is achieved without prior knowledge of the PSF, in contrast to the supervised approach used by DeepSTORM and the known PSF assumption in the Richardson-Lucy algorithm. Our method, PiMAE, provides a feasible scheme for achieving the hidden imaging mechanism in optical microscopy and has the potential to learn hidden mechanisms in many more systems. △ Less

Submitted 25 April, 2023; originally announced April 2023.

Label-Free Concept Bottleneck Models

Authors: Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, Tsui-Wei Weng

Abstract: Concept bottleneck models (CBM) are a popular way of creating more interpretable neural networks by having hidden layer neurons correspond to human-understandable concepts. However, existing CBMs and their variants have two crucial limitations: first, they need to collect labeled data for each of the predefined concepts, which is time consuming and labor intensive; second, the accuracy of a CBM is… ▽ More Concept bottleneck models (CBM) are a popular way of creating more interpretable neural networks by having hidden layer neurons correspond to human-understandable concepts. However, existing CBMs and their variants have two crucial limitations: first, they need to collect labeled data for each of the predefined concepts, which is time consuming and labor intensive; second, the accuracy of a CBM is often significantly lower than that of a standard neural network, especially on more complex datasets. This poor performance creates a barrier for adopting CBMs in practical real world applications. Motivated by these challenges, we propose Label-free CBM which is a novel framework to transform any neural network into an interpretable CBM without labeled concept data, while retaining a high accuracy. Our Label-free CBM has many advantages, it is: scalable - we present the first CBM scaled to ImageNet, efficient - creating a CBM takes only a few hours even for very large datasets, and automated - training it for a new dataset requires minimal human effort. Our code is available at https://github.com/Trustworthy-ML-Lab/Label-free-CBM. Finally, in Appendix B we conduct a large scale user evaluation of the interpretability of our method. △ Less

Submitted 5 June, 2023; v1 submitted 12 April, 2023; originally announced April 2023.

Comments: Published at ICLR 2023. New v2(5 June 2023): added crowdsourced human study in Appendix B

Constructive Assimilation: Boosting Contrastive Learning Performance through View Generation Strategies

Authors: Ligong Han, Seungwook Han, Shivchander Sudalairaj, Charlotte Loh, Rumen Dangovski, Fei Deng, Pulkit Agrawal, Dimitris Metaxas, Leonid Karlinsky, Tsui-Wei Weng, Akash Srivastava

Abstract: Transformations based on domain expertise (expert transformations), such as random-resized-crop and color-jitter, have proven critical to the success of contrastive learning techniques such as SimCLR. Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned. However for imagery data, so far none of these view-ge… ▽ More Transformations based on domain expertise (expert transformations), such as random-resized-crop and color-jitter, have proven critical to the success of contrastive learning techniques such as SimCLR. Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned. However for imagery data, so far none of these view-generation methods has been able to outperform expert transformations. In this work, we tackle a different question: instead of replacing expert transformations with generated views, can we constructively assimilate generated views with expert transformations? We answer this question in the affirmative and propose a view generation method and a simple, effective assimilation method that together improve the state-of-the-art by up to ~3.6% on three different datasets. Importantly, we conduct a detailed empirical study that systematically analyzes a range of view generation and assimilation methods and provides a holistic picture of the efficacy of learned views in contrastive representation learning. △ Less

Submitted 8 April, 2023; v1 submitted 2 April, 2023; originally announced April 2023.

Comments: Accepted at Generative Models for Computer Vision Workshop 2023

ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction

Authors: Wang Zhang, Tsui-Wei Weng, Subhro Das, Alexandre Megretski, Luca Daniel, Lam M. Nguyen

Abstract: Deep neural networks (DNN) have shown great capacity of modeling a dynamical system; nevertheless, they usually do not obey physics constraints such as conservation laws. This paper proposes a new learning framework named ConCerNet to improve the trustworthiness of the DNN based dynamics modeling to endow the invariant properties. ConCerNet consists of two steps: (i) a contrastive learning method… ▽ More Deep neural networks (DNN) have shown great capacity of modeling a dynamical system; nevertheless, they usually do not obey physics constraints such as conservation laws. This paper proposes a new learning framework named ConCerNet to improve the trustworthiness of the DNN based dynamics modeling to endow the invariant properties. ConCerNet consists of two steps: (i) a contrastive learning method to automatically capture the system invariants (i.e. conservation properties) along the trajectory observations; (ii) a neural projection layer to guarantee that the learned dynamics models preserve the learned invariants. We theoretically prove the functional relationship between the learned latent representation and the unknown system invariant function. Experiments show that our method consistently outperforms the baseline neural networks in both coordinate error and conservation metrics by a large margin. With neural network based parameterization and no dependence on prior knowledge, our method can be extended to complex and large-scale dynamics by leveraging an autoencoder. △ Less

Submitted 19 July, 2023; v1 submitted 11 February, 2023; originally announced February 2023.

Comments: Accepted by ICML 2023

Certified Interpretability Robustness for Class Activation Mapping

Authors: Alex Gu, Tsui-Wei Weng, Pin-Yu Chen, Sijia Liu, Luca Daniel

Abstract: Interpreting machine learning models is challenging but crucial for ensuring the safety of deep networks in autonomous driving systems. Due to the prevalence of deep learning based perception models in autonomous vehicles, accurately interpreting their predictions is crucial. While a variety of such methods have been proposed, most are shown to lack robustness. Yet, little has been done to provide… ▽ More Interpreting machine learning models is challenging but crucial for ensuring the safety of deep networks in autonomous driving systems. Due to the prevalence of deep learning based perception models in autonomous vehicles, accurately interpreting their predictions is crucial. While a variety of such methods have been proposed, most are shown to lack robustness. Yet, little has been done to provide certificates for interpretability robustness. Taking a step in this direction, we present CORGI, short for Certifiably prOvable Robustness Guarantees for Interpretability mapping. CORGI is an algorithm that takes in an input image and gives a certifiable lower bound for the robustness of the top k pixels of its CAM interpretability map. We show the effectiveness of CORGI via a case study on traffic sign data, certifying lower bounds on the minimum adversarial perturbation not far from (4-5x) state-of-the-art attack methods. △ Less

Submitted 26 January, 2023; originally announced January 2023.

Comments: 13 pages, 5 figures. Accepted to Machine Learning for Autonomous Driving Workshop at NeurIPS 2020

Neural Grasp Distance Fields for Robot Manipulation

Authors: Thomas Weng, David Held, Franziska Meier, Mustafa Mukadam

Abstract: We formulate grasp learning as a neural field and present Neural Grasp Distance Fields (NGDF). Here, the input is a 6D pose of a robot end effector and output is a distance to a continuous manifold of valid grasps for an object. In contrast to current approaches that predict a set of discrete candidate grasps, the distance-based NGDF representation is easily interpreted as a cost, and minimizing t… ▽ More We formulate grasp learning as a neural field and present Neural Grasp Distance Fields (NGDF). Here, the input is a 6D pose of a robot end effector and output is a distance to a continuous manifold of valid grasps for an object. In contrast to current approaches that predict a set of discrete candidate grasps, the distance-based NGDF representation is easily interpreted as a cost, and minimizing this cost produces a successful grasp pose. This grasp distance cost can be incorporated directly into a trajectory optimizer for joint optimization with other costs such as trajectory smoothness and collision avoidance. During optimization, as the various costs are balanced and minimized, the grasp target is allowed to smoothly vary, as the learned grasp field is continuous. We evaluate NGDF on joint grasp and motion planning in simulation and the real world, outperforming baselines by 63% execution success while generalizing to unseen query poses and unseen object shapes. Project page: https://sites.google.com/view/neural-grasp-distance-fields. △ Less

Submitted 28 December, 2023; v1 submitted 4 November, 2022; originally announced November 2022.

Comments: Accepted to ICRA 2023

Learning Sample Reweighting for Accuracy and Adversarial Robustness

Authors: Chester Holtz, Tsui-Wei Weng, Gal Mishne

Abstract: There has been great interest in enhancing the robustness of neural network classifiers to defend against adversarial perturbations through adversarial training, while balancing the trade-off between robust accuracy and standard accuracy. We propose a novel adversarial training framework that learns to reweight the loss associated with individual training samples based on a notion of class-conditi… ▽ More There has been great interest in enhancing the robustness of neural network classifiers to defend against adversarial perturbations through adversarial training, while balancing the trade-off between robust accuracy and standard accuracy. We propose a novel adversarial training framework that learns to reweight the loss associated with individual training samples based on a notion of class-conditioned margin, with the goal of improving robust generalization. We formulate weighted adversarial training as a bilevel optimization problem with the upper-level problem corresponding to learning a robust classifier, and the lower-level problem corresponding to learning a parametric function that maps from a sample's \textit{multi-class margin} to an importance weight. Extensive experiments demonstrate that our approach consistently improves both clean and robust accuracy compared to related methods and state-of-the-art baselines. △ Less

Submitted 20 October, 2022; originally announced October 2022.

Quantifying Safety of Learning-based Self-Driving Control Using Almost-Barrier Functions

Authors: Zhizhen Qin, Tsui-Wei Weng, Sicun Gao

Abstract: Path-tracking control of self-driving vehicles can benefit from deep learning for tackling longstanding challenges such as nonlinearity and uncertainty. However, deep neural controllers lack safety guarantees, restricting their practical use. We propose a new approach of learning almost-barrier functions, which approximately characterizes the forward invariant set for the system under neural contr… ▽ More Path-tracking control of self-driving vehicles can benefit from deep learning for tackling longstanding challenges such as nonlinearity and uncertainty. However, deep neural controllers lack safety guarantees, restricting their practical use. We propose a new approach of learning almost-barrier functions, which approximately characterizes the forward invariant set for the system under neural controllers, to quantitatively analyze the safety of deep neural controllers for path-tracking. We design sampling-based learning procedures for constructing candidate neural barrier functions, and certification procedures that utilize robustness analysis for neural networks to identify regions where the barrier conditions are fully satisfied. We use an adversarial training loop between learning and certification to optimize the almost-barrier functions. The learned barrier can also be used to construct online safety monitors through reachability analysis. We demonstrate effectiveness of our methods in quantifying safety of neural controllers in various simulation environments, ranging from simple kinematic models to the TORCS simulator with high-fidelity vehicle dynamics simulation. △ Less

Submitted 8 August, 2022; v1 submitted 28 July, 2022; originally announced July 2022.

Journal ref: International Conference on Intelligent Robots and Systems, IROS 2022, Kyoto, Japan, Oct 23 - Oct. 27, 2022

Learning to Singulate Layers of Cloth using Tactile Feedback

Authors: Sashank Tirumala, Thomas Weng, Daniel Seita, Oliver Kroemer, Zeynep Temel, David Held

Abstract: Robotic manipulation of cloth has applications ranging from fabrics manufacturing to handling blankets and laundry. Cloth manipulation is challenging for robots largely due to their high degrees of freedom, complex dynamics, and severe self-occlusions when in folded or crumpled configurations. Prior work on robotic manipulation of cloth relies primarily on vision sensors alone, which may pose chal… ▽ More Robotic manipulation of cloth has applications ranging from fabrics manufacturing to handling blankets and laundry. Cloth manipulation is challenging for robots largely due to their high degrees of freedom, complex dynamics, and severe self-occlusions when in folded or crumpled configurations. Prior work on robotic manipulation of cloth relies primarily on vision sensors alone, which may pose challenges for fine-grained manipulation tasks such as grasping a desired number of cloth layers from a stack of cloth. In this paper, we propose to use tactile sensing for cloth manipulation; we attach a tactile sensor (ReSkin) to one of the two fingertips of a Franka robot and train a classifier to determine whether the robot is grasping a specific number of cloth layers. During test-time experiments, the robot uses this classifier as part of its policy to grasp one or two cloth layers using tactile feedback to determine suitable grasping points. Experimental results over 180 physical trials suggest that the proposed method outperforms baselines that do not use tactile feedback and has better generalization to unseen cloth compared to methods that use image classifiers. Code, data, and videos are available at https://sites.google.com/view/reskin-cloth. △ Less

Submitted 22 July, 2022; originally announced July 2022.

Comments: IROS 2022. See https://sites.google.com/view/reskin-cloth for supplementary material

Fabric-GC: A Blockchain-based Gantt Chart System for Cross-organizational Project Management

Authors: Dun Li, Dezhi Han, Benhui Xia, Tien-Hsiung Weng, Arcangelo Castiglione, Kuan-Ching Li

Abstract: Large-scale production is always associated with more and more development and interaction among peers, and many fields achieve higher economic benefits through project cooperation. However, project managers in the traditional centralized approach cannot rearrange their activities to cross-organizational project management. Thanks to its characteristics, the Blockchain can represent a valid soluti… ▽ More Large-scale production is always associated with more and more development and interaction among peers, and many fields achieve higher economic benefits through project cooperation. However, project managers in the traditional centralized approach cannot rearrange their activities to cross-organizational project management. Thanks to its characteristics, the Blockchain can represent a valid solution to the problems mentioned above. In this article, we propose Fabric-GC, a Blockchain-based Gantt chart system. Fabric-GC enables to realize secure and effective cross-organizational cooperation for project management, providing access control to multiple parties for project visualization. Compared with other solutions, the proposed system is versatile, as it can be applied to project management in different fields and achieve effective and agile scheduling. Experimental results show that Fabric-GC achieves stable performance in large-scale request and processing distributed environments, where the data synchronization speed of the consortium chain reached four times faster than a public chain, achieving faster data consistency. △ Less

Submitted 19 July, 2022; originally announced July 2022.

CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks

Authors: Tuomas Oikarinen, Tsui-Wei Weng

Abstract: In this paper, we propose CLIP-Dissect, a new technique to automatically describe the function of individual hidden neurons inside vision networks. CLIP-Dissect leverages recent advances in multimodal vision/language models to label internal neurons with open-ended concepts without the need for any labeled data or human examples. We show that CLIP-Dissect provides more accurate descriptions than e… ▽ More In this paper, we propose CLIP-Dissect, a new technique to automatically describe the function of individual hidden neurons inside vision networks. CLIP-Dissect leverages recent advances in multimodal vision/language models to label internal neurons with open-ended concepts without the need for any labeled data or human examples. We show that CLIP-Dissect provides more accurate descriptions than existing methods for last layer neurons where the ground-truth is available as well as qualitatively good descriptions for hidden layer neurons. In addition, our method is very flexible: it is model agnostic, can easily handle new concepts and can be extended to take advantage of better multimodal models in the future. Finally CLIP-Dissect is computationally efficient and can label all neurons from five layers of ResNet-50 in just 4 minutes, which is more than 10 times faster than existing methods. Our code is available at https://github.com/Trustworthy-ML-Lab/CLIP-dissect. Finally, crowdsourced user study results are available at Appendix B to further support the effectiveness of our method. △ Less

Submitted 5 June, 2023; v1 submitted 22 April, 2022; originally announced April 2022.

Comments: Published in ICLR 2023 Conference (Spotlight). New v5(5 June 2023) - Added crowdsourced user study in Appendix B, not included in ICLR publication

Attacking c-MARL More Effectively: A Data Driven Approach

Authors: Nhan H. Pham, Lam M. Nguyen, Jie Chen, Hoang Thanh Lam, Subhro Das, Tsui-Wei Weng

Abstract: In recent years, a proliferation of methods were developed for cooperative multi-agent reinforcement learning (c-MARL). However, the robustness of c-MARL agents against adversarial attacks has been rarely explored. In this paper, we propose to evaluate the robustness of c-MARL agents via a model-based approach, named c-MBA. Our proposed formulation can craft much stronger adversarial state perturb… ▽ More In recent years, a proliferation of methods were developed for cooperative multi-agent reinforcement learning (c-MARL). However, the robustness of c-MARL agents against adversarial attacks has been rarely explored. In this paper, we propose to evaluate the robustness of c-MARL agents via a model-based approach, named c-MBA. Our proposed formulation can craft much stronger adversarial state perturbations of c-MARL agents to lower total team rewards than existing model-free approaches. In addition, we propose the first victim-agent selection strategy and the first data-driven approach to define targeted failure states where each of them allows us to develop even stronger adversarial attack without the expert knowledge to the underlying environment. Our numerical experiments on two representative MARL benchmarks illustrate the advantage of our approach over other baselines: our model-based attack consistently outperforms other baselines in all tested environments. △ Less

Submitted 10 September, 2023; v1 submitted 7 February, 2022; originally announced February 2022.

The relativistic 5f electronic structure of delocalized $α$-U and localized $δ$-Pu from the self consistent vertex corrected GW approach and X-ray Emission Spectroscopy

Authors: A. L. Kutepov, J. G. Tobin, S. -W. Yu, B. W. Chung, P. Roussel, S. Nowak, R. Alonso-Mori, T. Kroll, D. Nordlund, T. -C. Weng, D. Sokaras

Abstract: The recently developed self-consistent vertex corrected GW method is used to calculate the 5f electronic structure in delocalized $α$-U and localized $δ$-Pu, each of which is confirmed by the historical experimental approaches of direct and inverse photoemission. Tender X-Ray Emission Spectroscopy (XES), in a novel application to 5f electronic structure, is used to experimentally prove the existen… ▽ More The recently developed self-consistent vertex corrected GW method is used to calculate the 5f electronic structure in delocalized $α$-U and localized $δ$-Pu, each of which is confirmed by the historical experimental approaches of direct and inverse photoemission. Tender X-Ray Emission Spectroscopy (XES), in a novel application to 5f electronic structure, is used to experimentally prove the existence of 5f delocalization in $α$-U. △ Less

Submitted 16 November, 2021; originally announced November 2021.

Comments: 13 pages, 11 figures, 7 tables

On the Equivalence between Neural Network and Support Vector Machine

Authors: Yilan Chen, Wei Huang, Lam M. Nguyen, Tsui-Wei Weng

Abstract: Recent research shows that the dynamics of an infinitely wide neural network (NN) trained by gradient descent can be characterized by Neural Tangent Kernel (NTK) \citep{jacot2018neural}. Under the squared loss, the infinite-width NN trained by gradient descent with an infinitely small learning rate is equivalent to kernel regression with NTK \citep{arora2019exact}. However, the equivalence is only… ▽ More Recent research shows that the dynamics of an infinitely wide neural network (NN) trained by gradient descent can be characterized by Neural Tangent Kernel (NTK) \citep{jacot2018neural}. Under the squared loss, the infinite-width NN trained by gradient descent with an infinitely small learning rate is equivalent to kernel regression with NTK \citep{arora2019exact}. However, the equivalence is only known for ridge regression currently \citep{arora2019harnessing}, while the equivalence between NN and other kernel machines (KMs), e.g. support vector machine (SVM), remains unknown. Therefore, in this work, we propose to establish the equivalence between NN and SVM, and specifically, the infinitely wide NN trained by soft margin loss and the standard soft margin SVM with NTK trained by subgradient descent. Our main theoretical results include establishing the equivalences between NNs and a broad family of $\ell_2$ regularized KMs with finite-width bounds, which cannot be handled by prior work, and showing that every finite-width NN trained by such regularized loss functions is approximately a KM. Furthermore, we demonstrate our theory can enable three practical applications, including (i) \textit{non-vacuous} generalization bound of NN via the corresponding KM; (ii) \textit{non-trivial} robustness certificate for the infinite-width NN (while existing robustness verification methods would provide vacuous bounds); (iii) intrinsically more robust infinite-width NNs than those from previous kernel regression. Our code for the experiments is available at \url{https://github.com/leslie-CH/equiv-nn-svm}. △ Less

Submitted 7 July, 2022; v1 submitted 11 November, 2021; originally announced November 2021.

Comments: 35th Conference on Neural Information Processing Systems (NeurIPS 2021)

FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy

Authors: Thomas Weng, Sujay Bajracharya, Yufei Wang, Khush Agrawal, David Held

Abstract: We address the problem of goal-directed cloth manipulation, a challenging task due to the deformability of cloth. Our insight is that optical flow, a technique normally used for motion estimation in video, can also provide an effective representation for corresponding cloth poses across observation and goal images. We introduce FabricFlowNet (FFN), a cloth manipulation policy that leverages flow a… ▽ More We address the problem of goal-directed cloth manipulation, a challenging task due to the deformability of cloth. Our insight is that optical flow, a technique normally used for motion estimation in video, can also provide an effective representation for corresponding cloth poses across observation and goal images. We introduce FabricFlowNet (FFN), a cloth manipulation policy that leverages flow as both an input and as an action representation to improve performance. FabricFlowNet also elegantly switches between bimanual and single-arm actions based on the desired goal. We show that FabricFlowNet significantly outperforms state-of-the-art model-free and model-based cloth manipulation policies that take image input. We also present real-world experiments on a bimanual system, demonstrating effective sim-to-real transfer. Finally, we show that our method generalizes when trained on a single square cloth to other cloth shapes, such as T-shirts and rectangular cloths. Video and other supplementary materials are available at: https://sites.google.com/view/fabricflownet. △ Less

Submitted 10 April, 2022; v1 submitted 10 November, 2021; originally announced November 2021.

Comments: CoRL 2021

On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning

Authors: Ren Wang, Kaidi Xu, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Chuang Gan, Meng Wang

Abstract: Model-agnostic meta-learning (MAML) has emerged as one of the most successful meta-learning techniques in few-shot learning. It enables us to learn a meta-initialization} of model parameters (that we call meta-model) to rapidly adapt to new tasks using a small amount of labeled training data. Despite the generalization power of the meta-model, it remains elusive that how adversarial robustness can… ▽ More Model-agnostic meta-learning (MAML) has emerged as one of the most successful meta-learning techniques in few-shot learning. It enables us to learn a meta-initialization} of model parameters (that we call meta-model) to rapidly adapt to new tasks using a small amount of labeled training data. Despite the generalization power of the meta-model, it remains elusive that how adversarial robustness can be maintained by MAML in few-shot learning. In addition to generalization, robustness is also desired for a meta-model to defend adversarial examples (attacks). Toward promoting adversarial robustness in MAML, we first study WHEN a robustness-promoting regularization should be incorporated, given the fact that MAML adopts a bi-level (fine-tuning vs. meta-update) learning procedure. We show that robustifying the meta-update stage is sufficient to make robustness adapted to the task-specific fine-tuning stage even if the latter uses a standard training protocol. We also make additional justification on the acquired robustness adaptation by peering into the interpretability of neurons' activation maps. Furthermore, we investigate HOW robust regularization can efficiently be designed in MAML. We propose a general but easily-optimized robustness-regularized meta-learning framework, which allows the use of unlabeled data augmentation, fast adversarial attack generation, and computationally-light fine-tuning. In particular, we for the first time show that the auxiliary contrastive learning task can enhance the adversarial robustness of MAML. Finally, extensive experiments are conducted to demonstrate the effectiveness of our proposed methods in robust few-shot learning. △ Less

Submitted 20 February, 2021; originally announced February 2021.

Fast Training of Provably Robust Neural Networks by SingleProp

Authors: Akhilan Boopathy, Tsui-Wei Weng, Sijia Liu, Pin-Yu Chen, Gaoyuan Zhang, Luca Daniel

Abstract: Recent works have developed several methods of defending neural networks against adversarial attacks with certified guarantees. However, these techniques can be computationally costly due to the use of certification during training. We develop a new regularizer that is both more efficient than existing certified defenses, requiring only one additional forward propagation through a network, and can… ▽ More Recent works have developed several methods of defending neural networks against adversarial attacks with certified guarantees. However, these techniques can be computationally costly due to the use of certification during training. We develop a new regularizer that is both more efficient than existing certified defenses, requiring only one additional forward propagation through a network, and can be used to train networks with similar certified accuracy. Through experiments on MNIST and CIFAR-10 we demonstrate improvements in training speed and comparable certified accuracy compared to state-of-the-art certified defenses. △ Less

Submitted 1 February, 2021; originally announced February 2021.

Comments: Published at AAAI 2021

Higher-Order Certification for Randomized Smoothing

Authors: Jeet Mohapatra, Ching-Yun Ko, Tsui-Wei Weng, Pin-Yu Chen, Sijia Liu, Luca Daniel

Abstract: Randomized smoothing is a recently proposed defense against adversarial attacks that has achieved SOTA provable robustness against $\ell_2$ perturbations. A number of publications have extended the guarantees to other metrics, such as $\ell_1$ or $\ell_\infty$, by using different smoothing measures. Although the current framework has been shown to yield near-optimal $\ell_p$ radii, the total safet… ▽ More Randomized smoothing is a recently proposed defense against adversarial attacks that has achieved SOTA provable robustness against $\ell_2$ perturbations. A number of publications have extended the guarantees to other metrics, such as $\ell_1$ or $\ell_\infty$, by using different smoothing measures. Although the current framework has been shown to yield near-optimal $\ell_p$ radii, the total safety region certified by the current framework can be arbitrarily small compared to the optimal. In this work, we propose a framework to improve the certified safety region for these smoothed classifiers without changing the underlying smoothing scheme. The theoretical contributions are as follows: 1) We generalize the certification for randomized smoothing by reformulating certified radius calculation as a nested optimization problem over a class of functions. 2) We provide a method to calculate the certified safety region using $0^{th}$-order and $1^{st}$-order information for Gaussian-smoothed classifiers. We also provide a framework that generalizes the calculation for certification using higher-order information. 3) We design efficient, high-confidence estimators for the relevant statistics of the first-order information. Combining the theoretical contribution 2) and 3) allows us to certify safety region that are significantly larger than the ones provided by the current methods. On CIFAR10 and Imagenet datasets, the new regions certified by our approach achieve significant improvements on general $\ell_1$ certified radii and on the $\ell_2$ certified radii for color-space attacks ($\ell_2$ restricted to 1 channel) while also achieving smaller improvements on the general $\ell_2$ certified radii. Our framework can also provide a way to circumvent the current impossibility results on achieving higher magnitude of certified radii without requiring the use of data-dependent smoothing techniques. △ Less

Submitted 13 October, 2020; originally announced October 2020.

Comments: Accepted to NeurIPS2020(spotlight)

Cloth Region Segmentation for Robust Grasp Selection

Authors: Jianing Qian, Thomas Weng, Luxin Zhang, Brian Okorn, David Held

Abstract: Cloth detection and manipulation is a common task in domestic and industrial settings, yet such tasks remain a challenge for robots due to cloth deformability. Furthermore, in many cloth-related tasks like laundry folding and bed making, it is crucial to manipulate specific regions like edges and corners, as opposed to folds. In this work, we focus on the problem of segmenting and grasping these k… ▽ More Cloth detection and manipulation is a common task in domestic and industrial settings, yet such tasks remain a challenge for robots due to cloth deformability. Furthermore, in many cloth-related tasks like laundry folding and bed making, it is crucial to manipulate specific regions like edges and corners, as opposed to folds. In this work, we focus on the problem of segmenting and grasping these key regions. Our approach trains a network to segment the edges and corners of a cloth from a depth image, distinguishing such regions from wrinkles or folds. We also provide a novel algorithm for estimating the grasp location, direction, and directional uncertainty from the segmentation. We demonstrate our method on a real robot system and show that it outperforms baseline methods on grasping success. Video and other supplementary materials are available at: https://sites.google.com/view/cloth-segmentation. △ Less

Submitted 12 August, 2020; originally announced August 2020.

Comments: Accepted at IROS 2020. The first two authors contributed equally and are listed in alphabetical order

Journal ref: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Robust Deep Reinforcement Learning through Adversarial Loss

Authors: Tuomas Oikarinen, Wang Zhang, Alexandre Megretski, Luca Daniel, Tsui-Wei Weng

Abstract: Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent's inputs, which raises concerns about deploying such agents in the real world. To address this issue, we propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against $l_p$-norm bounded adversarial attacks. Our framewor… ▽ More Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent's inputs, which raises concerns about deploying such agents in the real world. To address this issue, we propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against $l_p$-norm bounded adversarial attacks. Our framework is compatible with popular deep reinforcement learning algorithms and we demonstrate its performance with deep Q-learning, A3C and PPO. We experiment on three deep RL benchmarks (Atari, MuJoCo and ProcGen) to show the effectiveness of our robust training algorithm. Our RADIAL-RL agents consistently outperform prior methods when tested against attacks of varying strength and are more computationally efficient to train. In addition, we propose a new evaluation method called Greedy Worst-Case Reward (GWC) to measure attack agnostic robustness of deep RL agents. We show that GWC can be evaluated efficiently and is a good estimate of the reward under the worst possible sequence of adversarial attacks. All code used for our experiments is available at https://github.com/tuomaso/radial_rl_v2. △ Less

Submitted 10 November, 2021; v1 submitted 5 August, 2020; originally announced August 2020.

Multi-modal Transfer Learning for Grasping Transparent and Specular Objects

Authors: Thomas Weng, Amith Pallankize, Yimin Tang, Oliver Kroemer, David Held

Abstract: State-of-the-art object grasping methods rely on depth sensing to plan robust grasps, but commercially available depth sensors fail to detect transparent and specular objects. To improve grasping performance on such objects, we introduce a method for learning a multi-modal perception model by bootstrapping from an existing uni-modal model. This transfer learning approach requires only a pre-existi… ▽ More State-of-the-art object grasping methods rely on depth sensing to plan robust grasps, but commercially available depth sensors fail to detect transparent and specular objects. To improve grasping performance on such objects, we introduce a method for learning a multi-modal perception model by bootstrapping from an existing uni-modal model. This transfer learning approach requires only a pre-existing uni-modal grasping model and paired multi-modal image data for training, foregoing the need for ground-truth grasp success labels nor real grasp attempts. Our experiments demonstrate that our approach is able to reliably grasp transparent and reflective objects. Video and supplementary material are available at https://sites.google.com/view/transparent-specular-grasping. △ Less

Submitted 29 May, 2020; originally announced June 2020.

Comments: RA-L with presentation at ICRA 2020

Journal ref: IEEE ROBOTICS AND AUTOMATION LETTERS, VOL. 5, NO. 3, JULY 2020. 3791-3798

Verification of Neural Network Control Policy Under Persistent Adversarial Perturbation

Authors: Yuh-Shyang Wang, Tsui-Wei Weng, Luca Daniel

Abstract: Deep neural networks are known to be fragile to small adversarial perturbations. This issue becomes more critical when a neural network is interconnected with a physical system in a closed loop. In this paper, we show how to combine recent works on neural network certification tools (which are mainly used in static settings such as image classification) with robust control theory to certify a neur… ▽ More Deep neural networks are known to be fragile to small adversarial perturbations. This issue becomes more critical when a neural network is interconnected with a physical system in a closed loop. In this paper, we show how to combine recent works on neural network certification tools (which are mainly used in static settings such as image classification) with robust control theory to certify a neural network policy in a control loop. Specifically, we give a sufficient condition and an algorithm to ensure that the closed loop state and control constraints are satisfied when the persistent adversarial perturbation is l-infinity norm bounded. Our method is based on finding a positively invariant set of the closed loop dynamical system, and thus we do not require the differentiability or the continuity of the neural network policy. Along with the verification result, we also develop an effective attack strategy for neural network control systems that outperforms exhaustive Monte-Carlo search significantly. We show that our certification algorithm works well on learned models and achieves 5 times better result than the traditional Lipschitz-based method to certify the robustness of a neural network policy on a cart pole control problem. △ Less

Submitted 17 August, 2019; originally announced August 2019.

Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective

Authors: Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Mingyi Hong, Xue Lin

Abstract: Graph neural networks (GNNs) which apply the deep neural networks to graph data have achieved significant performance for the task of semi-supervised node classification. However, only few work has addressed the adversarial robustness of GNNs. In this paper, we first present a novel gradient-based attack method that facilitates the difficulty of tackling discrete graph data. When comparing to curr… ▽ More Graph neural networks (GNNs) which apply the deep neural networks to graph data have achieved significant performance for the task of semi-supervised node classification. However, only few work has addressed the adversarial robustness of GNNs. In this paper, we first present a novel gradient-based attack method that facilitates the difficulty of tackling discrete graph data. When comparing to current adversarial attacks on GNNs, the results show that by only perturbing a small number of edge perturbations, including addition and deletion, our optimization-based attack can lead to a noticeable decrease in classification performance. Moreover, leveraging our gradient-based attack, we propose the first optimization-based adversarial training for GNNs. Our method yields higher robustness against both different gradient based and greedy attack methods without sacrificing classification accuracy on original graph. △ Less

Submitted 14 October, 2019; v1 submitted 10 June, 2019; originally announced June 2019.

Comments: Accepted by IJCAI 2019, the 28th International Joint Conference on Artificial Intelligence

Journal ref: International Joint Conference on Artificial Intelligence (IJCAI-2019)

POPQORN: Quantifying Robustness of Recurrent Neural Networks

Authors: Ching-Yun Ko, Zhaoyang Lyu, Tsui-Wei Weng, Luca Daniel, Ngai Wong, Dahua Lin

Abstract: The vulnerability to adversarial attacks has been a critical issue for deep neural networks. Addressing this issue requires a reliable way to evaluate the robustness of a network. Recently, several methods have been developed to compute $\textit{robustness quantification}$ for neural networks, namely, certified lower bounds of the minimum adversarial perturbation. Such methods, however, were devis… ▽ More The vulnerability to adversarial attacks has been a critical issue for deep neural networks. Addressing this issue requires a reliable way to evaluate the robustness of a network. Recently, several methods have been developed to compute $\textit{robustness quantification}$ for neural networks, namely, certified lower bounds of the minimum adversarial perturbation. Such methods, however, were devised for feed-forward networks, e.g. multi-layer perceptron or convolutional networks. It remains an open problem to quantify robustness for recurrent networks, especially LSTM and GRU. For such networks, there exist additional challenges in computing the robustness quantification, such as handling the inputs at multiple steps and the interaction between gates and states. In this work, we propose $\textit{POPQORN}$ ($\textbf{P}$ropagated-$\textbf{o}$ut$\textbf{p}$ut $\textbf{Q}$uantified R$\textbf{o}$bustness for $\textbf{RN}$Ns), a general algorithm to quantify robustness of RNNs, including vanilla RNNs, LSTMs, and GRUs. We demonstrate its effectiveness on different network architectures and show that the robustness quantification on individual steps can lead to new insights. △ Less

Submitted 17 May, 2019; originally announced May 2019.

Comments: 10 pages, Ching-Yun Ko and Zhaoyang Lyu contributed equally, accepted to ICML 2019. Please see arXiv source codes for the appendix by clicking [Other formats]

What retards the response of graphene based gaseous sensor

Authors: Hui-Fen Zhang, Bo-Yuan Ning, Tsu-Chien Weng, Dong-Ping Wu, Xi-Jing Ning

Abstract: Graphene based sensor to gas molecules should be ultrasensitive and ultrafast because of the single-atomic thickness of graphene, while the response is not fast. Usually, the measured response time for many molecules, such as CO, NH3, SO2, CO2 and NO2 and so on, is on the scale of minutes or longer. In the present work, we found via \emph{ab initio} calculations there exists a potential barrier la… ▽ More Graphene based sensor to gas molecules should be ultrasensitive and ultrafast because of the single-atomic thickness of graphene, while the response is not fast. Usually, the measured response time for many molecules, such as CO, NH3, SO2, CO2 and NO2 and so on, is on the scale of minutes or longer. In the present work, we found via \emph{ab initio} calculations there exists a potential barrier larger than 0.7 eV that hinders the gas molecule to land directly at the defective sites of graphene and retards the response. An efficient approach to the problem is suggested as modifying the graphene sheet with other molecules to reduce the potential barrier and was demonstrated by a graphene sheet modified by Fe2O3 molecules that shows fast response to H2S molecule, and the calculated response time is close to the measured one, 500 $μ$s. △ Less

Submitted 20 February, 2019; originally announced February 2019.

Comments: 23 pages, 9 figures

Journal ref: Sensors and Actuators A: Physical, 295(2019) 188-192

Comparison of two efficient methods for calculating partition functions

Authors: Le-Cheng Gong, Bo-Yuan Ning, Tsu-Chien Weng, Xi-Jing Ning

Abstract: In the long-time pursuit of the solution to calculate the partition function (or free energy) of condensed matter, Monte-Carlo-based nested sampling should be the state-of-the-art method, and very recently, we established a direct integral approach that works at least four orders faster. In present work, the above two methods were applied to solid argon at temperatures up to $300$K, and the derive… ▽ More In the long-time pursuit of the solution to calculate the partition function (or free energy) of condensed matter, Monte-Carlo-based nested sampling should be the state-of-the-art method, and very recently, we established a direct integral approach that works at least four orders faster. In present work, the above two methods were applied to solid argon at temperatures up to $300$K, and the derived internal energy and pressure were compared with the molecular dynamics simulation as well as experimental measurements, showing that the calculation precision of our approach is about 10 times higher than that of the nested sampling method. △ Less

Submitted 19 February, 2019; originally announced February 2019.

Comments: 6 pages, 4 figures

Searching for the optimum conditions for silicene growth by calculations of the free energy

Authors: Yu-Peng Liu, Bo-Yuan Ning, Le-Cheng Gong, Tsu-Chien Weng, Xi-Jing Ning

Abstract: Very recently we developed an efficient method to calculate the free energy of 2D materials on substrates and achieved high calculation precision for graphene or $γ$-graphyne on copper substrates. In the present work, the method was further confirmed to be accurate by molecular dynamic simulations of silicene on Ag substrate using empirical potential and was applied to predict the optimum conditio… ▽ More Very recently we developed an efficient method to calculate the free energy of 2D materials on substrates and achieved high calculation precision for graphene or $γ$-graphyne on copper substrates. In the present work, the method was further confirmed to be accurate by molecular dynamic simulations of silicene on Ag substrate using empirical potential and was applied to predict the optimum conditions based on \emph{ab initio} calculations for silicene growth on Ag (110) and Ag (111) surface, which are in good agreement with previous experimental observations. △ Less

Submitted 17 February, 2019; originally announced February 2019.

Comments: 11 pages, 4 figures

Journal ref: Nanomaterials, 9(2019) 978

Calculating the free energy of 2D materials on substrates

Authors: Yu-Peng Liu, Bo-Yuan Ning, Le-Cheng Gong, Tsu-Chien Weng, Xi-Jing Ning

Abstract: A method was developed to calculate the free energy of 2D materials on substrates and was demonstrated by the system of graphene and γ-graphyne on copper substrate. The method works at least 3 orders faster than state-of-the-art algorithms, and the accuracy was tested by molecular dynamics simulations, showing that the precision for calculations of the internal energy achieves up to 0.03% in a tem… ▽ More A method was developed to calculate the free energy of 2D materials on substrates and was demonstrated by the system of graphene and γ-graphyne on copper substrate. The method works at least 3 orders faster than state-of-the-art algorithms, and the accuracy was tested by molecular dynamics simulations, showing that the precision for calculations of the internal energy achieves up to 0.03% in a temperature range from 100 to 1300K. As expected, the calculated the free energy of a graphene sheet on Cu (111) or Ni (111) surface in a temperature range up to 3000K is always smaller than the one of a γ-graphyne sheet with the same number of C atoms, which is consistent with the fact that growth of graphene on the substrates is much easier than γ-graphyne. △ Less

Submitted 26 January, 2019; originally announced January 2019.

Comments: 16 pages, 4 figures

Journal ref: Nanomaterials, 9(2019) 978

Solution to the key problem of statistical physics -- calculations of partition function of many-body systems

Authors: Bo-Yuan Ning, Le-Cheng Gong, Tsu-Chien Weng, Xi-Jing Ning

Abstract: The key problem of statistical physics standing over one hundred years is how to exactly calculate the partition function (or free energy) of many-body interaction systems, which severely hinders application of the theory for realistic systems. Here we present a novel approach that works at least four orders faster than state-of-the-art algorithms to the problem and can be applied to predict therm… ▽ More The key problem of statistical physics standing over one hundred years is how to exactly calculate the partition function (or free energy) of many-body interaction systems, which severely hinders application of the theory for realistic systems. Here we present a novel approach that works at least four orders faster than state-of-the-art algorithms to the problem and can be applied to predict thermal properties of large molecules or macroscopic condensed matters via \emph{ab initio} calculations.The method was demonstrated by C$_{60}$ molecules, solid and liquid copper (up to $\sim 600$GPa), solid argon, graphene and silicene on substrate, and the derived internal energy or pressure is in a good agreement with the results of vast molecular dynamics simulations in a temperature range up to $2500$K, achieving a precision at least one order higher than previous methods. And, for the first time, the realistic isochoric equation of state for solid argon was reproduced directly from the partition function. △ Less

Submitted 25 April, 2019; v1 submitted 23 January, 2019; originally announced January 2019.

Comments: Request of supplementary information should be addressed to BYN and XJN

Finite-Sum Smooth Optimization with SARAH

Authors: Lam M. Nguyen, Marten van Dijk, Dzung T. Phan, Phuong Ha Nguyen, Tsui-Wei Weng, Jayant R. Kalagnanam

Abstract: The total complexity (measured as the total number of gradient computations) of a stochastic first-order optimization algorithm that finds a first-order stationary point of a finite-sum smooth nonconvex objective function $F(w)=\frac{1}{n} \sum_{i=1}^n f_i(w)$ has been proven to be at least $Ω(\sqrt{n}/ε)$ for $n \leq \mathcal{O}(ε^{-2})$ where $ε$ denotes the attained accuracy… ▽ More The total complexity (measured as the total number of gradient computations) of a stochastic first-order optimization algorithm that finds a first-order stationary point of a finite-sum smooth nonconvex objective function $F(w)=\frac{1}{n} \sum_{i=1}^n f_i(w)$ has been proven to be at least $Ω(\sqrt{n}/ε)$ for $n \leq \mathcal{O}(ε^{-2})$ where $ε$ denotes the attained accuracy $\mathbb{E}[ \|\nabla F(\tilde{w})\|^2] \leq ε$ for the outputted approximation $\tilde{w}$ (Fang et al., 2018). In this paper, we provide a convergence analysis for a slightly modified version of the SARAH algorithm (Nguyen et al., 2017a;b) and achieve total complexity that matches the lower-bound worst case complexity in (Fang et al., 2018) up to a constant factor when $n \leq \mathcal{O}(ε^{-2})$ for nonconvex problems. For convex optimization, we propose SARAH++ with sublinear convergence for general convex and linear convergence for strongly convex problems; and we provide a practical version for which numerical experiments on various datasets show an improved performance. △ Less

Submitted 22 April, 2019; v1 submitted 22 January, 2019; originally announced January 2019.

PROVEN: Certifying Robustness of Neural Networks with a Probabilistic Approach

Authors: Tsui-Wei Weng, Pin-Yu Chen, Lam M. Nguyen, Mark S. Squillante, Ivan Oseledets, Luca Daniel

Abstract: With deep neural networks providing state-of-the-art machine learning models for numerous machine learning tasks, quantifying the robustness of these models has become an important area of research. However, most of the research literature merely focuses on the \textit{worst-case} setting where the input of the neural network is perturbed with noises that are constrained within an $\ell_p$ ball; a… ▽ More With deep neural networks providing state-of-the-art machine learning models for numerous machine learning tasks, quantifying the robustness of these models has become an important area of research. However, most of the research literature merely focuses on the \textit{worst-case} setting where the input of the neural network is perturbed with noises that are constrained within an $\ell_p$ ball; and several algorithms have been proposed to compute certified lower bounds of minimum adversarial distortion based on such worst-case analysis. In this paper, we address these limitations and extend the approach to a \textit{probabilistic} setting where the additive noises can follow a given distributional characterization. We propose a novel probabilistic framework PROVEN to PRObabilistically VErify Neural networks with statistical guarantees -- i.e., PROVEN certifies the probability that the classifier's top-1 prediction cannot be altered under any constrained $\ell_p$ norm perturbation to a given input. Importantly, we show that it is possible to derive closed-form probabilistic certificates based on current state-of-the-art neural network robustness verification frameworks. Hence, the probabilistic certificates provided by PROVEN come naturally and with almost no overhead when obtaining the worst-case certified lower bounds from existing methods such as Fast-Lin, CROWN and CNN-Cert. Experiments on small and large MNIST and CIFAR neural network models demonstrate our probabilistic approach can achieve up to around $75\%$ improvement in the robustness certification with at least a $99.99\%$ confidence compared with the worst-case robustness certificate delivered by CROWN. △ Less

Submitted 7 January, 2019; v1 submitted 18 December, 2018; originally announced December 2018.

Comments: updated ref [25]