-
Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation
Authors:
Jessica Quaye,
Alicia Parrish,
Oana Inel,
Charvi Rastogi,
Hannah Rose Kirk,
Minsuk Kahng,
Erin van Liemt,
Max Bartolo,
Jess Tsang,
Justin White,
Nathan Clement,
Rafael Mosquera,
Juan Ciro,
Vijay Janapa Reddi,
Lora Aroyo
Abstract:
With the rise of text-to-image (T2I) generative AI models reaching wide audiences, it is critical to evaluate model robustness against non-obvious attacks to mitigate the generation of offensive images. By focusing on ``implicitly adversarial'' prompts (those that trigger T2I models to generate unsafe images for non-obvious reasons), we isolate a set of difficult safety issues that human creativit…
▽ More
With the rise of text-to-image (T2I) generative AI models reaching wide audiences, it is critical to evaluate model robustness against non-obvious attacks to mitigate the generation of offensive images. By focusing on ``implicitly adversarial'' prompts (those that trigger T2I models to generate unsafe images for non-obvious reasons), we isolate a set of difficult safety issues that human creativity is well-suited to uncover. To this end, we built the Adversarial Nibbler Challenge, a red-teaming methodology for crowdsourcing a diverse set of implicitly adversarial prompts. We have assembled a suite of state-of-the-art T2I models, employed a simple user interface to identify and annotate harms, and engaged diverse populations to capture long-tail safety issues that may be overlooked in standard testing. The challenge is run in consecutive rounds to enable a sustained discovery and analysis of safety pitfalls in T2I models.
In this paper, we present an in-depth account of our methodology, a systematic study of novel attack strategies and discussion of safety failures revealed by challenge participants. We also release a companion visualization tool for easy exploration and derivation of insights from the dataset. The first challenge round resulted in over 10k prompt-image pairs with machine annotations for safety. A subset of 1.5k samples contains rich human annotations of harm types and attack styles. We find that 14% of images that humans consider harmful are mislabeled as ``safe'' by machines. We have identified new attack strategies that highlight the complexity of ensuring T2I model robustness. Our findings emphasize the necessity of continual auditing and adaptation as new vulnerabilities emerge. We are confident that this work will enable proactive, iterative safety assessments and promote responsible development of T2I models.
△ Less
Submitted 13 May, 2024; v1 submitted 14 February, 2024;
originally announced March 2024.
-
An Encoding Framework for Binarized Images using HyperDimensional Computing
Authors:
Laura Smets,
Werner Van Leekwijck,
Ing Jyh Tsang,
Steven Latré
Abstract:
Hyperdimensional Computing (HDC) is a brain-inspired and light-weight machine learning method. It has received significant attention in the literature as a candidate to be applied in the wearable internet of things, near-sensor artificial intelligence applications and on-device processing. HDC is computationally less complex than traditional deep learning algorithms and typically achieves moderate…
▽ More
Hyperdimensional Computing (HDC) is a brain-inspired and light-weight machine learning method. It has received significant attention in the literature as a candidate to be applied in the wearable internet of things, near-sensor artificial intelligence applications and on-device processing. HDC is computationally less complex than traditional deep learning algorithms and typically achieves moderate to good classification performance. A key aspect that determines the performance of HDC is the encoding of the input data to the hyperdimensional (HD) space. This article proposes a novel light-weight approach relying only on native HD arithmetic vector operations to encode binarized images that preserves similarity of patterns at nearby locations by using point of interest selection and local linear mapping. The method reaches an accuracy of 97.35% on the test set for the MNIST data set and 84.12% for the Fashion-MNIST data set. These results outperform other studies using baseline HDC with different encoding approaches and are on par with more complex hybrid HDC models. The proposed encoding approach also demonstrates a higher robustness to noise and blur compared to the baseline encoding.
△ Less
Submitted 1 December, 2023;
originally announced December 2023.
-
The Trifecta: Three simple techniques for training deeper Forward-Forward networks
Authors:
Thomas Dooms,
Ing Jyh Tsang,
Jose Oramas
Abstract:
Modern machine learning models are able to outperform humans on a variety of non-trivial tasks. However, as the complexity of the models increases, they consume significant amounts of power and still struggle to generalize effectively to unseen data. Local learning, which focuses on updating subsets of a model's parameters at a time, has emerged as a promising technique to address these issues. Re…
▽ More
Modern machine learning models are able to outperform humans on a variety of non-trivial tasks. However, as the complexity of the models increases, they consume significant amounts of power and still struggle to generalize effectively to unseen data. Local learning, which focuses on updating subsets of a model's parameters at a time, has emerged as a promising technique to address these issues. Recently, a novel local learning algorithm, called Forward-Forward, has received widespread attention due to its innovative approach to learning. Unfortunately, its application has been limited to smaller datasets due to scalability issues. To this end, we propose The Trifecta, a collection of three simple techniques that synergize exceptionally well and drastically improve the Forward-Forward algorithm on deeper networks. Our experiments demonstrate that our models are on par with similarly structured, backpropagation-based models in both training speed and test accuracy on simple datasets. This is achieved by the ability to learn representations that are informative locally, on a layer-by-layer basis, and retain their informativeness when propagated to deeper layers in the architecture. This leads to around 84% accuracy on CIFAR-10, a notable improvement (25%) over the original FF algorithm. These results highlight the potential of Forward-Forward as a genuine competitor to backpropagation and as a promising research avenue.
△ Less
Submitted 12 December, 2023; v1 submitted 29 November, 2023;
originally announced November 2023.
-
Co-learning synaptic delays, weights and adaptation in spiking neural networks
Authors:
Lucas Deckers,
Laurens Van Damme,
Ing Jyh Tsang,
Werner Van Leekwijck,
Steven Latré
Abstract:
Spiking neural networks (SNN) distinguish themselves from artificial neural networks (ANN) because of their inherent temporal processing and spike-based computations, enabling a power-efficient implementation in neuromorphic hardware. In this paper, we demonstrate that data processing with spiking neurons can be enhanced by co-learning the connection weights with two other biologically inspired ne…
▽ More
Spiking neural networks (SNN) distinguish themselves from artificial neural networks (ANN) because of their inherent temporal processing and spike-based computations, enabling a power-efficient implementation in neuromorphic hardware. In this paper, we demonstrate that data processing with spiking neurons can be enhanced by co-learning the connection weights with two other biologically inspired neuronal features: 1) a set of parameters describing neuronal adaptation processes and 2) synaptic propagation delays. The former allows the spiking neuron to learn how to specifically react to incoming spikes based on its past. The trained adaptation parameters result in neuronal heterogeneity, which is found in the brain and also leads to a greater variety in available spike patterns. The latter enables to learn to explicitly correlate patterns that are temporally distanced. Synaptic delays reflect the time an action potential requires to travel from one neuron to another. We show that each of the co-learned features separately leads to an improvement over the baseline SNN and that the combination of both leads to state-of-the-art SNN results on all speech recognition datasets investigated with a simple 2-hidden layer feed-forward network. Our SNN outperforms the ANN on the neuromorpic datasets (Spiking Heidelberg Digits and Spiking Speech Commands), even with fewer trainable parameters. On the 35-class Google Speech Commands dataset, our SNN also outperforms a GRU of similar size. Our work presents brain-inspired improvements to SNN that enable them to excel over an equivalent ANN of similar size on tasks with rich temporal dynamics.
△ Less
Submitted 12 September, 2023;
originally announced November 2023.
-
Training a HyperDimensional Computing Classifier using a Threshold on its Confidence
Authors:
Laura Smets,
Werner Van Leekwijck,
Ing Jyh Tsang,
Steven Latre
Abstract:
Hyperdimensional computing (HDC) has become popular for light-weight and energy-efficient machine learning, suitable for wearable Internet-of-Things (IoT) devices and near-sensor or on-device processing. HDC is computationally less complex than traditional deep learning algorithms and achieves moderate to good classification performance. This article proposes to extend the training procedure in HD…
▽ More
Hyperdimensional computing (HDC) has become popular for light-weight and energy-efficient machine learning, suitable for wearable Internet-of-Things (IoT) devices and near-sensor or on-device processing. HDC is computationally less complex than traditional deep learning algorithms and achieves moderate to good classification performance. This article proposes to extend the training procedure in HDC by taking into account not only wrongly classified samples, but also samples that are correctly classified by the HDC model but with low confidence. As such, a confidence threshold is introduced that can be tuned for each dataset to achieve the best classification accuracy. The proposed training procedure is tested on UCIHAR, CTG, ISOLET and HAND dataset for which the performance consistently improves compared to the baseline across a range of confidence threshold values. The extended training procedure also results in a shift towards higher confidence values of the correctly classified samples making the classifier not only more accurate but also more confident about its predictions.
△ Less
Submitted 30 November, 2023; v1 submitted 30 May, 2023;
originally announced May 2023.
-
Research Focused Software Development Kits and Wearable Devices in Physical Activity Research
Authors:
Jason Tsang,
Harry Prapavessis
Abstract:
Introduction: The Canadian Guidelines recommend physical activity for overall health benefits, including cognitive, emotional, functional, and physical health. However, traditional research methods are inefficient and outdated. This paper aims to guide researchers in enhancing their research methods using software development kits and wearable smart devices. Methods: A generic model application wa…
▽ More
Introduction: The Canadian Guidelines recommend physical activity for overall health benefits, including cognitive, emotional, functional, and physical health. However, traditional research methods are inefficient and outdated. This paper aims to guide researchers in enhancing their research methods using software development kits and wearable smart devices. Methods: A generic model application was transformed into a research-based mobile application based on the UCLA researchers who collaborated with Apple. First, the research question and goals were identified. Then, three open-source software development kits (SDKs) were used to modify the generic model into the desired application. ResearchKit was used for informed consent, surveys, and active tasks. CareKit was the protocol manager to create participant protocols and track progress. Finally, HealthKit was used to access and share health-related data. The content expert evaluated the application, and the participant experience was optimized for easy use. The collected health-related data were analyzed to identify any significant findings. Results: Wearable health devices offer a convenient and non-invasive way to monitor and track health-related information. Conclusion: Leveraging the data provided by wearable devices, researchers can gain insights into the effectiveness of interventions and inform the development of evidence-based physical activity guidelines. The use of software development kits and wearable devices can enhance research methods and provide valuable insights into overall health benefits.
△ Less
Submitted 12 May, 2023;
originally announced May 2023.
-
Hybrid Reward Architecture for Reinforcement Learning
Authors:
Harm van Seijen,
Mehdi Fatemi,
Joshua Romoff,
Romain Laroche,
Tavian Barnes,
Jeffrey Tsang
Abstract:
One of the main challenges in reinforcement learning (RL) is generalisation. In typical deep RL methods this is achieved by approximating the optimal value function with a low-dimensional representation using a deep network. While this approach works well in many domains, in domains where the optimal value function cannot easily be reduced to a low-dimensional representation, learning can be very…
▽ More
One of the main challenges in reinforcement learning (RL) is generalisation. In typical deep RL methods this is achieved by approximating the optimal value function with a low-dimensional representation using a deep network. While this approach works well in many domains, in domains where the optimal value function cannot easily be reduced to a low-dimensional representation, learning can be very slow and unstable. This paper contributes towards tackling such challenging domains, by proposing a new method, called Hybrid Reward Architecture (HRA). HRA takes as input a decomposed reward function and learns a separate value function for each component reward function. Because each component typically only depends on a subset of all features, the corresponding value function can be approximated more easily by a low-dimensional representation, enabling more effective learning. We demonstrate HRA on a toy-problem and the Atari game Ms. Pac-Man, where HRA achieves above-human performance.
△ Less
Submitted 27 November, 2017; v1 submitted 13 June, 2017;
originally announced June 2017.
-
The parametrized probabilistic finite-state transducer probe game player fingerprint model
Authors:
Jeffrey Tsang
Abstract:
Fingerprinting operators generate functional signatures of game players and are useful for their automated analysis independent of representation or encoding. The theory for a fingerprinting operator which returns the length-weighted probability of a given move pair occurring from playing the investigated agent against a general parametrized probabilistic finite-state transducer (PFT) is developed…
▽ More
Fingerprinting operators generate functional signatures of game players and are useful for their automated analysis independent of representation or encoding. The theory for a fingerprinting operator which returns the length-weighted probability of a given move pair occurring from playing the investigated agent against a general parametrized probabilistic finite-state transducer (PFT) is developed, applicable to arbitrary iterated games. Results for the distinguishing power of the 1-state opponent model, uniform approximability of fingerprints of arbitrary players, analyticity and Lipschitz continuity of fingerprints for logically possible players, and equicontinuity of the fingerprints of bounded-state probabilistic transducers are derived. Algorithms for the efficient computation of special instances are given; the shortcomings of a previous model, strictly generalized here from a simple projection of the new model, are explained in terms of regularity condition violations, and the extra power and functional niceness of the new fingerprints demonstrated. The 2-state deterministic finite-state transducers (DFTs) are fingerprinted and pairwise distances computed; using this the structure of DFTs in strategy space is elucidated.
△ Less
Submitted 28 January, 2014;
originally announced January 2014.
