In the Emotion Recognition in Conversation task, recent investigations have utilized attention mechanisms exploring relationships among utterances from intra- and inter-speakers for modeling emotional interaction between them. However, attributes such as speaker personality traits remain unexplored and present challenges in terms of their applicability to other tasks or compatibility with diverse model architectures. Therefore, this work introduces a novel framework named BiosERC, which investigates speaker characteristics in a conversation. By employing Large Language Models (LLMs), we extract the ``biographical information'' of the speaker within a conversation as supplementary knowledge injected into the model to classify emotional labels for each utterance. Our proposed method achieved state-of-the-art (SOTA) results on three famous benchmark datasets: IEMOCAP, MELD, and EmoryNLP, demonstrating the effectiveness and generalization of our model and showcasing its potential for adaptation to various conversation analysis tasks.

Performance comparison between our proposed method and previous works on the test sets.

unzip the file data.zip to extract data.

• IEMOCAP Data structure examples:

  {
      # this is first conversation 
      "Ses05M_impro03": { 
          "labels": [
          4,
          2,
          4,
          4 
          ],
          "sentences": [
          "Guess what?",
          "what?",
          "I did it, I asked her to marry me.",
          "Yes, I did it."
          ], 
          "genders": [
          "M",
          "F",
          "M",
          "M",
          "F", 
          ]
      },
  
      # this is second conversation 
      "Ses05M_impro03": { 
          "labels": [
          4,
          2,
          ],
          "sentences": [
          "Guess what?",
          "what?", 
          ], 
          "genders": [
          "M",
          "F",  
          ]
      }
  }

Init python environment

    conda create --prefix=./env_py38  python=3.9
    conda activate ./env_py38 
    pip install -r requirements.txt

1. Init environment follow the above step.
2. Data peprocessing.
   1. Put all the raw data to the folder data/. The overview of data structure:

          .
          ├── data/
          │   ├── llm_vectors/  # save speaker description data
          │   │   ├── meld.valid_spdescV2_Llama-2-70b-chat-hf.json            
          │   │   ├── meld.train_spdescV2_Llama-2-70b-chat-hf.json 
          │   │   ├── meld.test_spdescV2_Llama-2-70b-chat-hf.json
          │   │   └── ...
          │   ├── meld.test.json
          │   ├── meld.train.json
          │   ├── meld.valid.json
          │   ├── ...
          │   ├── iemocap.test.json
          │   ├── iemocap.train.json
          │   └── iemocap.valid.json
          ├── src/
          ├── finetuned_llm/
          └── ...
      

3. Train
   Run following command to train a new model.

   python src/llm_bio_extract.py # to extract speaker bio
   bash scrips/train_llm.sh # to train a llm model

   Note: Please check this scripts to check the setting and choose which data you want to run.

    @InProceedings{bioserc,
    author="Xue, Jieying
    and Nguyen, Minh-Phuong
    and Matheny, Blake
    and Nguyen, Le-Minh",
    editor="[updating]",
    title="BiosERC: Integrating Biography Speakers Supported by LLMs for ERC Tasks",
    booktitle="Artificial Neural Networks and Machine Learning -- ICANN 2024",
    year="2024",
    publisher="Springer Nature Switzerland",
    address="Cham",
    pages="[updating]", 
    isbn="[updating]"
    }