-
Continual Learning in the Presence of Repetition
Authors:
Hamed Hemati,
Lorenzo Pellegrini,
Xiaotian Duan,
Zixuan Zhao,
Fangfang Xia,
Marc Masana,
Benedikt Tscheschner,
Eduardo Veas,
Yuxiang Zheng,
Shiji Zhao,
Shao-Yuan Li,
Sheng-Jun Huang,
Vincenzo Lomonaco,
Gido M. van de Ven
Abstract:
Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the st…
▽ More
Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design.
△ Less
Submitted 7 May, 2024;
originally announced May 2024.
-
Continual Learning and Catastrophic Forgetting
Authors:
Gido M. van de Ven,
Nicholas Soures,
Dhireesha Kudithipudi
Abstract:
This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data. Although continual learning is a natural skill for the human brain, it is very challenging for artificial neural networks. An important reason is that, when learning something new, these networks tend to quickly and drastically forget what they had…
▽ More
This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data. Although continual learning is a natural skill for the human brain, it is very challenging for artificial neural networks. An important reason is that, when learning something new, these networks tend to quickly and drastically forget what they had learned before, a phenomenon known as catastrophic forgetting. Especially in the last decade, continual learning has become an extensively studied topic in deep learning. This book chapter reviews the insights that this field has generated.
△ Less
Submitted 8 March, 2024;
originally announced March 2024.
-
Infinite dSprites for Disentangled Continual Learning: Separating Memory Edits from Generalization
Authors:
Sebastian Dziadzio,
Çağatay Yıldız,
Gido M. van de Ven,
Tomasz Trzciński,
Tinne Tuytelaars,
Matthias Bethge
Abstract:
The ability of machine learning systems to learn continually is hindered by catastrophic forgetting, the tendency of neural networks to overwrite previously acquired knowledge when learning a new task. Existing methods mitigate this problem through regularization, parameter isolation, or rehearsal, but they are typically evaluated on benchmarks comprising only a handful of tasks. In contrast, huma…
▽ More
The ability of machine learning systems to learn continually is hindered by catastrophic forgetting, the tendency of neural networks to overwrite previously acquired knowledge when learning a new task. Existing methods mitigate this problem through regularization, parameter isolation, or rehearsal, but they are typically evaluated on benchmarks comprising only a handful of tasks. In contrast, humans are able to learn over long time horizons in dynamic, open-world environments, effortlessly memorizing unfamiliar objects and reliably recognizing them under various transformations. To make progress towards closing this gap, we introduce Infinite dSprites, a parsimonious tool for creating continual classification and disentanglement benchmarks of arbitrary length and with full control over generative factors. We show that over a sufficiently long time horizon, the performance of all major types of continual learning methods deteriorates on this simple benchmark. This result highlights an important and previously overlooked aspect of continual learning: given a finite modelling capacity and an arbitrarily long learning horizon, efficient learning requires memorizing class-specific information and accumulating knowledge about general mechanisms. In a simple setting with direct supervision on the generative factors, we show how learning class-agnostic transformations offers a way to circumvent catastrophic forgetting and improve classification accuracy over time. Our approach sets the stage for continual learning over hundreds of tasks with explicit control over memorization and forgetting, emphasizing open-set classification and one-shot generalization.
△ Less
Submitted 29 July, 2024; v1 submitted 27 December, 2023;
originally announced December 2023.
-
Continual Learning of Diffusion Models with Generative Distillation
Authors:
Sergi Masip,
Pau Rodriguez,
Tinne Tuytelaars,
Gido M. van de Ven
Abstract:
Diffusion models are powerful generative models that achieve state-of-the-art performance in image synthesis. However, training them demands substantial amounts of data and computational resources. Continual learning would allow for incrementally learning new tasks and accumulating knowledge, thus enabling the reuse of trained models for further learning. One potentially suitable continual learnin…
▽ More
Diffusion models are powerful generative models that achieve state-of-the-art performance in image synthesis. However, training them demands substantial amounts of data and computational resources. Continual learning would allow for incrementally learning new tasks and accumulating knowledge, thus enabling the reuse of trained models for further learning. One potentially suitable continual learning approach is generative replay, where a copy of a generative model trained on previous tasks produces synthetic data that are interleaved with data from the current task. However, standard generative replay applied to diffusion models results in a catastrophic loss in denoising capabilities. In this paper, we propose generative distillation, an approach that distils the entire reverse process of a diffusion model. We demonstrate that our approach substantially improves the continual learning performance of generative replay with only a modest increase in the computational costs.
△ Less
Submitted 20 May, 2024; v1 submitted 23 November, 2023;
originally announced November 2023.
-
Continual Learning: Applications and the Road Forward
Authors:
Eli Verwimp,
Rahaf Aljundi,
Shai Ben-David,
Matthias Bethge,
Andrea Cossu,
Alexander Gepperth,
Tyler L. Hayes,
Eyke Hüllermeier,
Christopher Kanan,
Dhireesha Kudithipudi,
Christoph H. Lampert,
Martin Mundt,
Razvan Pascanu,
Adrian Popescu,
Andreas S. Tolias,
Joost van de Weijer,
Bing Liu,
Vincenzo Lomonaco,
Tinne Tuytelaars,
Gido M. van de Ven
Abstract:
Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past. In this work, we take a step back, and ask: "Why should one care about continual learning in the first place?". We set the stage by examining recent continual learning papers published at four…
▽ More
Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past. In this work, we take a step back, and ask: "Why should one care about continual learning in the first place?". We set the stage by examining recent continual learning papers published at four major machine learning conferences, and show that memory-constrained settings dominate the field. Then, we discuss five open problems in machine learning, and even though they might seem unrelated to continual learning at first sight, we show that continual learning will inevitably be part of their solution. These problems are model editing, personalization and specialization, on-device learning, faster (re-)training and reinforcement learning. Finally, by comparing the desiderata from these unsolved problems and the current assumptions in continual learning, we highlight and discuss four future directions for continual learning research. We hope that this work offers an interesting perspective on the future of continual learning, while displaying its potential value and the paths we have to pursue in order to make it successful. This work is the result of the many discussions the authors had at the Dagstuhl seminar on Deep Continual Learning, in March 2023.
△ Less
Submitted 28 March, 2024; v1 submitted 20 November, 2023;
originally announced November 2023.
-
Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How
Authors:
Timm Hess,
Tinne Tuytelaars,
Gido M. van de Ven
Abstract:
Recent years have seen considerable progress in the continual training of deep neural networks, predominantly thanks to approaches that add replay or regularization terms to the loss function to approximate the joint loss over all tasks so far. However, we show that even with a perfect approximation to the joint loss, these approaches still suffer from temporary but substantial forgetting when sta…
▽ More
Recent years have seen considerable progress in the continual training of deep neural networks, predominantly thanks to approaches that add replay or regularization terms to the loss function to approximate the joint loss over all tasks so far. However, we show that even with a perfect approximation to the joint loss, these approaches still suffer from temporary but substantial forgetting when starting to train on a new task. Motivated by this 'stability gap', we propose that continual learning strategies should focus not only on the optimization objective, but also on the way this objective is optimized. While there is some continual learning work that alters the optimization trajectory (e.g., using gradient projection techniques), this line of research is positioned as alternative to improving the optimization objective, while we argue it should be complementary. In search of empirical support for our proposition, we perform a series of pre-registered experiments combining replay-approximated joint objectives with gradient projection-based optimization routines. However, this first experimental attempt fails to show clear and consistent benefits. Nevertheless, our conceptual arguments, as well as some of our empirical results, demonstrate the distinctive importance of the optimization trajectory in continual learning, thereby opening up a new direction for continual learning research.
△ Less
Submitted 21 June, 2024; v1 submitted 8 November, 2023;
originally announced November 2023.
-
Prediction Error-based Classification for Class-Incremental Learning
Authors:
Michał Zając,
Tinne Tuytelaars,
Gido M. van de Ven
Abstract:
Class-incremental learning (CIL) is a particularly challenging variant of continual learning, where the goal is to learn to discriminate between all classes presented in an incremental fashion. Existing approaches often suffer from excessive forgetting and imbalance of the scores assigned to classes that have not been seen together during training. In this study, we introduce a novel approach, Pre…
▽ More
Class-incremental learning (CIL) is a particularly challenging variant of continual learning, where the goal is to learn to discriminate between all classes presented in an incremental fashion. Existing approaches often suffer from excessive forgetting and imbalance of the scores assigned to classes that have not been seen together during training. In this study, we introduce a novel approach, Prediction Error-based Classification (PEC), which differs from traditional discriminative and generative classification paradigms. PEC computes a class score by measuring the prediction error of a model trained to replicate the outputs of a frozen random neural network on data from that class. The method can be interpreted as approximating a classification rule based on Gaussian Process posterior variance. PEC offers several practical advantages, including sample efficiency, ease of tuning, and effectiveness even when data are presented one class at a time. Our empirical results show that PEC performs strongly in single-pass-through-data CIL, outperforming other rehearsal-free baselines in all cases and rehearsal-based methods with moderate replay buffer size in most cases across multiple benchmarks.
△ Less
Submitted 9 March, 2024; v1 submitted 30 May, 2023;
originally announced May 2023.
-
Knowledge Accumulation in Continually Learned Representations and the Issue of Feature Forgetting
Authors:
Timm Hess,
Eli Verwimp,
Gido M. van de Ven,
Tinne Tuytelaars
Abstract:
Continual learning research has shown that neural networks suffer from catastrophic forgetting "at the output level", but it is debated whether this is also the case at the level of learned representations. Multiple recent studies ascribe representations a certain level of innate robustness against forgetting -- that they only forget minimally in comparison with forgetting at the output level. We…
▽ More
Continual learning research has shown that neural networks suffer from catastrophic forgetting "at the output level", but it is debated whether this is also the case at the level of learned representations. Multiple recent studies ascribe representations a certain level of innate robustness against forgetting -- that they only forget minimally in comparison with forgetting at the output level. We revisit and expand upon the experiments that revealed this difference in forgetting and illustrate the coexistence of two phenomena that affect the quality of continually learned representations: knowledge accumulation and feature forgetting. Taking both aspects into account, we show that, even though forgetting in the representation (i.e. feature forgetting) can be small in absolute terms, when measuring relative to how much was learned during a task, forgetting in the representation tends to be just as catastrophic as forgetting at the output level. Next we show that this feature forgetting is problematic as it substantially slows down the incremental learning of good general representations (i.e. knowledge accumulation). Finally, we study how feature forgetting and knowledge accumulation are affected by different types of continual learning methods.
△ Less
Submitted 24 June, 2024; v1 submitted 3 April, 2023;
originally announced April 2023.
-
Natural continual learning: success is a journey, not (just) a destination
Authors:
Ta-Chu Kao,
Kristopher T. Jensen,
Gido M. van de Ven,
Alberto Bernacchia,
Guillaume Hennequin
Abstract:
Biological agents are known to learn many different tasks over the course of their lives, and to be able to revisit previous tasks and behaviors with little to no loss in performance. In contrast, artificial agents are prone to 'catastrophic forgetting' whereby performance on previous tasks deteriorates rapidly as new ones are acquired. This shortcoming has recently been addressed using methods th…
▽ More
Biological agents are known to learn many different tasks over the course of their lives, and to be able to revisit previous tasks and behaviors with little to no loss in performance. In contrast, artificial agents are prone to 'catastrophic forgetting' whereby performance on previous tasks deteriorates rapidly as new ones are acquired. This shortcoming has recently been addressed using methods that encourage parameters to stay close to those used for previous tasks. This can be done by (i) using specific parameter regularizers that map out suitable destinations in parameter space, or (ii) guiding the optimization journey by projecting gradients into subspaces that do not interfere with previous tasks. However, these methods often exhibit subpar performance in both feedforward and recurrent neural networks, with recurrent networks being of interest to the study of neural dynamics supporting biological continual learning. In this work, we propose Natural Continual Learning (NCL), a new method that unifies weight regularization and projected gradient descent. NCL uses Bayesian weight regularization to encourage good performance on all tasks at convergence and combines this with gradient projection using the prior precision, which prevents catastrophic forgetting during optimization. Our method outperforms both standard weight regularization techniques and projection based approaches when applied to continual learning problems in feedforward and recurrent networks. Finally, the trained networks evolve task-specific dynamics that are strongly preserved as new tasks are learned, similar to experimental findings in biological circuits.
△ Less
Submitted 15 December, 2021; v1 submitted 15 June, 2021;
originally announced June 2021.
-
Class-Incremental Learning with Generative Classifiers
Authors:
Gido M. van de Ven,
Zhe Li,
Andreas S. Tolias
Abstract:
Incrementally training deep neural networks to recognize new classes is a challenging problem. Most existing class-incremental learning methods store data or use generative replay, both of which have drawbacks, while 'rehearsal-free' alternatives such as parameter regularization or bias-correction methods do not consistently achieve high performance. Here, we put forward a new strategy for class-i…
▽ More
Incrementally training deep neural networks to recognize new classes is a challenging problem. Most existing class-incremental learning methods store data or use generative replay, both of which have drawbacks, while 'rehearsal-free' alternatives such as parameter regularization or bias-correction methods do not consistently achieve high performance. Here, we put forward a new strategy for class-incremental learning: generative classification. Rather than directly learning the conditional distribution p(y|x), our proposal is to learn the joint distribution p(x,y), factorized as p(x|y)p(y), and to perform classification using Bayes' rule. As a proof-of-principle, here we implement this strategy by training a variational autoencoder for each class to be learned and by using importance sampling to estimate the likelihoods p(x|y). This simple approach performs very well on a diverse set of continual learning benchmarks, outperforming generative replay and other existing baselines that do not store data.
△ Less
Submitted 28 April, 2021; v1 submitted 20 April, 2021;
originally announced April 2021.
-
Energy-Based Models for Continual Learning
Authors:
Shuang Li,
Yilun Du,
Gido M. van de Ven,
Igor Mordatch
Abstract:
We motivate Energy-Based Models (EBMs) as a promising model class for continual learning problems. Instead of tackling continual learning via the use of external memory, growing models, or regularization, EBMs change the underlying training objective to cause less interference with previously learned information. Our proposed version of EBMs for continual learning is simple, efficient, and outperf…
▽ More
We motivate Energy-Based Models (EBMs) as a promising model class for continual learning problems. Instead of tackling continual learning via the use of external memory, growing models, or regularization, EBMs change the underlying training objective to cause less interference with previously learned information. Our proposed version of EBMs for continual learning is simple, efficient, and outperforms baseline methods by a large margin on several benchmarks. Moreover, our proposed contrastive divergence-based training objective can be combined with other continual learning methods, resulting in substantial boosts in their performance. We further show that EBMs are adaptable to a more general continual learning setting where the data distribution changes without the notion of explicitly delineated tasks. These observations point towards EBMs as a useful building block for future continual learning methods.
△ Less
Submitted 18 December, 2022; v1 submitted 24 November, 2020;
originally announced November 2020.
-
A Simple Lifelong Learning Approach
Authors:
Joshua T. Vogelstein,
Jayanta Dey,
Hayden S. Helm,
Will LeVine,
Ronak D. Mehta,
Tyler M. Tomita,
Haoyin Xu,
Ali Geisa,
Qingyang Wang,
Gido M. van de Ven,
Chenyu Gao,
Weiwei Yang,
Bryan Tower,
Jonathan Larson,
Christopher M. White,
Carey E. Priebe
Abstract:
In lifelong learning, data are used to improve performance not only on the present task, but also on past and future (unencountered) tasks. While typical transfer learning algorithms can improve performance on future tasks, their performance on prior tasks degrades upon learning new tasks (called forgetting). Many recent approaches for continual or lifelong learning have attempted to maintain perf…
▽ More
In lifelong learning, data are used to improve performance not only on the present task, but also on past and future (unencountered) tasks. While typical transfer learning algorithms can improve performance on future tasks, their performance on prior tasks degrades upon learning new tasks (called forgetting). Many recent approaches for continual or lifelong learning have attempted to maintain performance on old tasks given new tasks. But striving to avoid forgetting sets the goal unnecessarily low. The goal of lifelong learning should be to use data to improve performance on both future tasks (forward transfer) and past tasks (backward transfer). In this paper, we show that a simple approach -- representation ensembling -- demonstrates both forward and backward transfer in a variety of simulated and benchmark data scenarios, including tabular, vision (CIFAR-100, 5-dataset, Split Mini-Imagenet, and Food1k), and speech (spoken digit), in contrast to various reference algorithms, which typically failed to transfer either forward or backward, or both. Moreover, our proposed approach can flexibly operate with or without a computational budget.
△ Less
Submitted 11 June, 2024; v1 submitted 27 April, 2020;
originally announced April 2020.
-
Three scenarios for continual learning
Authors:
Gido M. van de Ven,
Andreas S. Tolias
Abstract:
Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning. In recent years, numerous methods have been proposed for continual learning, but due to differences in evaluation protocols it is difficult to directly compare their performance. To enable more structured comparisons, we describe thr…
▽ More
Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning. In recent years, numerous methods have been proposed for continual learning, but due to differences in evaluation protocols it is difficult to directly compare their performance. To enable more structured comparisons, we describe three continual learning scenarios based on whether at test time task identity is provided and--in case it is not--whether it must be inferred. Any sequence of well-defined tasks can be performed according to each scenario. Using the split and permuted MNIST task protocols, for each scenario we carry out an extensive comparison of recently proposed continual learning methods. We demonstrate substantial differences between the three scenarios in terms of difficulty and in terms of how efficient different methods are. In particular, when task identity must be inferred (i.e., class incremental learning), we find that regularization-based approaches (e.g., elastic weight consolidation) fail and that replaying representations of previous experiences seems required for solving this scenario.
△ Less
Submitted 15 April, 2019;
originally announced April 2019.
-
Generative replay with feedback connections as a general strategy for continual learning
Authors:
Gido M. van de Ven,
Andreas S. Tolias
Abstract:
A major obstacle to developing artificial intelligence applications capable of true lifelong learning is that artificial neural networks quickly or catastrophically forget previously learned tasks when trained on a new one. Numerous methods for alleviating catastrophic forgetting are currently being proposed, but differences in evaluation protocols make it difficult to directly compare their perfo…
▽ More
A major obstacle to developing artificial intelligence applications capable of true lifelong learning is that artificial neural networks quickly or catastrophically forget previously learned tasks when trained on a new one. Numerous methods for alleviating catastrophic forgetting are currently being proposed, but differences in evaluation protocols make it difficult to directly compare their performance. To enable more meaningful comparisons, here we identified three distinct scenarios for continual learning based on whether task identity is known and, if it is not, whether it needs to be inferred. Performing the split and permuted MNIST task protocols according to each of these scenarios, we found that regularization-based approaches (e.g., elastic weight consolidation) failed when task identity needed to be inferred. In contrast, generative replay combined with distillation (i.e., using class probabilities as "soft targets") achieved superior performance in all three scenarios. Addressing the issue of efficiency, we reduced the computational cost of generative replay by integrating the generative model into the main model by equipping it with generative feedback or backward connections. This Replay-through-Feedback approach substantially shortened training time with no or negligible loss in performance. We believe this to be an important first step towards making the powerful technique of generative replay scalable to real-world continual learning applications.
△ Less
Submitted 17 April, 2019; v1 submitted 27 September, 2018;
originally announced September 2018.
-
Stability and folds in an elastocapillary system
Authors:
Amir Akbari,
Reghan J. Hill,
Theo G. M. van de Ven
Abstract:
We examine the equilibrium and stability of an elastocapillary system to model drying-induced structural failures. The model comprises a circular elastic membrane with a hole at the center that is deformed by the capillary pressure of simply connected and doubly connected menisci. Using variational and spectral methods, stability is related to the slope of equilibrium branches in the liquid conten…
▽ More
We examine the equilibrium and stability of an elastocapillary system to model drying-induced structural failures. The model comprises a circular elastic membrane with a hole at the center that is deformed by the capillary pressure of simply connected and doubly connected menisci. Using variational and spectral methods, stability is related to the slope of equilibrium branches in the liquid content versus pressure diagram for the constrained and unconstrained problems. The second-variation spectra are separately determined for the membrane and meniscus, showing that the membrane out-of-plane spectrum and the in-plane spectrum at large elatocapillary numbers are both positive, so that only meniscus perturbations can cause instability. At small elastocapillary numbers, the in-plane spectrum has a negative eigenvalue, inducing wrinkling instabilities in thin membranes. In contrast, the smallest eigenvalue of the meniscus spectrum always changes sign at a pressure turning point where stability exchange occurs in the unconstrained problem. We also examine configurations in which the meniscus and membrane are individually stable, while the elastocapillary system as a whole is not; this emphasizes the connection between stability and the coupling of elastic and capillary forces.
△ Less
Submitted 27 May, 2015;
originally announced May 2015.
-
Catenoid stability with a free contact line
Authors:
Amir Akbari,
Reghan J. Hill,
Theo G. M. van de Ven
Abstract:
Contact-drop dispensing is central to many small-scale applications, such as direct-scanning probe lithography and micromachined fountain-pen techniques. Accurate and controllable dispensing required for nanometer-resolved surface patterning hinges on the stability and breakup of liquid bridges. Here, we analytically study the stability of catenoids pinned at one contact line with the other free t…
▽ More
Contact-drop dispensing is central to many small-scale applications, such as direct-scanning probe lithography and micromachined fountain-pen techniques. Accurate and controllable dispensing required for nanometer-resolved surface patterning hinges on the stability and breakup of liquid bridges. Here, we analytically study the stability of catenoids pinned at one contact line with the other free to move on a substrate subject to axisymmetric and non-axisymmetric perturbations. We apply a variational formulation to derive the corresponding stability criteria. The maximal stability region and stability region are represented in the favourable and canonical phase diagrams, providing a complete description of catenoid equilibrium and stability. All catenoids are stable with respect to non-axisymmetric perturbations. For a fixed contact angle, there exists a critical volume below which catenoids are unstable to axisymmetric perturbations. Equilibrium solution multiplicity is discussed in detail, and we elucidate how geometrical symmetry is reflected in the maximal stability and stability regions.
△ Less
Submitted 26 May, 2015;
originally announced May 2015.