This repository contains the code for DPO of LLM-jp models.

See pyproject.toml for the required packages.

poetry install
poetry shell

Here is the command to train a model using 8 GPUs.

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 accelerate launch --config_file accelerate_configs/zero2.yaml train.py