SwitchCIT: Switching for Continual Instruction Tuning of Large Language Models
Authors:
Xinbo Wu,
Max Hartman,
Vidhata Arjun Jayaraman,
Lav R. Varshney
Abstract:
Large language models (LLMs) have exhibited impressive capabilities in various domains, particularly in general language understanding. However these models, trained on massive text data, may not be finely optimized for specific tasks triggered by instructions. Continual instruction tuning is crucial to adapt LLMs to evolving tasks and domains, ensuring their effectiveness and relevance across a w…
▽ More
Large language models (LLMs) have exhibited impressive capabilities in various domains, particularly in general language understanding. However these models, trained on massive text data, may not be finely optimized for specific tasks triggered by instructions. Continual instruction tuning is crucial to adapt LLMs to evolving tasks and domains, ensuring their effectiveness and relevance across a wide range of applications. In the context of continual instruction tuning, where models are sequentially trained on different tasks, catastrophic forgetting can occur, leading to performance degradation on previously learned tasks. This work addresses the catastrophic forgetting in continual instruction learning for LLMs through a switching mechanism for routing computations to parameter-efficient tuned models. We demonstrate the effectiveness of our method through experiments on continual instruction tuning of different natural language generation tasks.
△ Less
Submitted 16 July, 2024;
originally announced July 2024.
Comparing Network Structures of Commercial and Non-commercial Biohacking Online-communities
Authors:
Sipra Bihani,
Michael Hartman,
Florian Sobiegalla,
Amanda Rosenberg
Abstract:
This paper compares two biohacking groups, Bulletproof Executive and DIYbio, whose distinct goals result in differences in social network structures, activities and entry points.
This paper compares two biohacking groups, Bulletproof Executive and DIYbio, whose distinct goals result in differences in social network structures, activities and entry points.
△ Less
Submitted 10 March, 2015;
originally announced March 2015.
