Understanding Alignment in Multimodal LLMs: A Comprehensive Study
Authors:
Elmira Amirloo,
Jean-Philippe Fauconnier,
Christoph Roesmann,
Christian Kerl,
Rinu Boney,
Yusu Qian,
Zirui Wang,
Afshin Dehghan,
Yinfei Yang,
Zhe Gan,
Peter Grasch
Abstract:
Preference alignment has become a crucial component in enhancing the performance of Large Language Models (LLMs), yet its impact in Multimodal Large Language Models (MLLMs) remains comparatively underexplored. Similar to language models, MLLMs for image understanding tasks encounter challenges like hallucination. In MLLMs, hallucination can occur not only by stating incorrect facts but also by pro…
▽ More
Preference alignment has become a crucial component in enhancing the performance of Large Language Models (LLMs), yet its impact in Multimodal Large Language Models (MLLMs) remains comparatively underexplored. Similar to language models, MLLMs for image understanding tasks encounter challenges like hallucination. In MLLMs, hallucination can occur not only by stating incorrect facts but also by producing responses that are inconsistent with the image content. A primary objective of alignment for MLLMs is to encourage these models to align responses more closely with image information. Recently, multiple works have introduced preference datasets for MLLMs and examined different alignment methods, including Direct Preference Optimization (DPO) and Proximal Policy Optimization (PPO). However, due to variations in datasets, base model types, and alignment methods, it remains unclear which specific elements contribute most significantly to the reported improvements in these works. In this paper, we independently analyze each aspect of preference alignment in MLLMs. We start by categorizing the alignment algorithms into two groups, offline (such as DPO), and online (such as online-DPO), and show that combining offline and online methods can improve the performance of the model in certain scenarios. We review a variety of published multimodal preference datasets and discuss how the details of their construction impact model performance. Based on these insights, we introduce a novel way of creating multimodal preference data called Bias-Driven Hallucination Sampling (BDHS) that needs neither additional annotation nor external models, and show that it can achieve competitive performance to previously published alignment work for multimodal models across a range of benchmarks.
△ Less
Submitted 2 July, 2024;
originally announced July 2024.
Online Motion Planning based on Nonlinear Model Predictive Control with Non-Euclidean Rotation Groups
Authors:
Christoph Rösmann,
Artemi Makarow,
Torsten Bertram
Abstract:
This paper proposes a novel online motion planning approach to robot navigation based on nonlinear model predictive control. Common approaches rely on pure Euclidean optimization parameters. In robot navigation, however, state spaces often include rotational components which span over non-Euclidean rotation groups. The proposed approach applies nonlinear increment and difference operators in the e…
▽ More
This paper proposes a novel online motion planning approach to robot navigation based on nonlinear model predictive control. Common approaches rely on pure Euclidean optimization parameters. In robot navigation, however, state spaces often include rotational components which span over non-Euclidean rotation groups. The proposed approach applies nonlinear increment and difference operators in the entire optimization scheme to explicitly consider these groups. Realizations include but are not limited to quadratic form and time-optimal objectives. A complex parking scenario for the kinematic bicycle model demonstrates the effectiveness and practical relevance of the approach. In case of simpler robots (e.g. differential drive), a comparative analysis in a hierarchical planning setting reveals comparable computation times and performance. The approach is available in a modular and highly configurable open-source C++ software framework.
△ Less
Submitted 5 June, 2020;
originally announced June 2020.
