![](https://cdn-avatars.huggingface.co/v1/production/uploads/6566ad47dee7d2c57a9dd125/8plPO2Ig_G5Vy6NyvOrf9.png)
![InfiMM's profile picture](https://cdn-avatars.huggingface.co/v1/production/uploads/6566ad47dee7d2c57a9dd125/8plPO2Ig_G5Vy6NyvOrf9.png)
InfiMM
AI & ML interests
None defined yet.
InfiMM
InfiMM, inspired by the Flamingo architecture, sets itself apart with unique training data and diverse large language models (LLMs). This approach allows InfiMM to maintain the core strengths of Flamingo while offering enhanced capabilities. As the premier open-sourced variant in this domain, InfiMM excels in accessibility and adaptability, driven by community collaboration. It's more than an emulation of Flamingo; it's an innovation in visual language processing.
Our model is another attempt to produce the result reported in the paper "Flamingo: A Large-scale Visual Language Model for Multimodal Understanding" by DeepMind. Compared with previous open-sourced attempts (OpenFlamingo and IDEFIC), InfiMM offers a more flexible models, allowing for a wide range of applications. In particular, InfiMM integrates the latest LLM models into VLM domain the reveals the impact of LLMs with different scales and architectures.
Please note that InfiMM is currently in beta stage and we are continuously working on improving it.
News
- 🎉 [2024.08.15] Our paper was accepted by ACL 2023 InfiMM.
- 🎉 [2024.03.02] We release the InfiMM-HD.
- 🎉 [2024.01.11] We release the first set of MLLMs we trained InfiMM-Zephyr, InfiMM-LLaMA13B and InfiMM-Vicuna13B.
- 🎉 [2024.01.10] We release a survey about Multimodal Large Language Models (MLLMs) reasoning capability at here.
- 🎉 [2023.11.18] We release InfiMM-Eval at here, an Open-ended VQA benchmark dataset specifically designed for MLLMs, with a focus on complex reasoning tasks. The leaderboard can be found via Papers with Code or project page.
Citation
@inproceedings{liu-etal-2024-infimm,
title = "{I}nfi{MM}: Advancing Multimodal Understanding with an Open-Sourced Visual Language Model",
author = "Liu, Haogeng and
You, Quanzeng and
Wang, Yiqi and
Han, Xiaotian and
Zhai, Bohan and
Liu, Yongfei and
Chen, Wentao and
Jian, Yiren and
Tao, Yunzhe and
Yuan, Jianbo and
He, Ran and
Yang, Hongxia",
editor = "Ku, Lun-Wei and
Martins, Andre and
Srikumar, Vivek",
booktitle = "Findings of the Association for Computational Linguistics ACL 2024",
month = aug,
year = "2024",
address = "Bangkok, Thailand and virtual meeting",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.findings-acl.27",
pages = "485--492",
abstract = "In this work, we present InfiMM, an advanced Multimodal Large Language Model that adapts to intricate vision-language tasks. InfiMM, inspired by the Flamingo architecture, distinguishes itself through the utilization of large-scale training data, comprehensive training strategies, and diverse large language models. This approach ensures the preservation of Flamingo{'}s foundational strengths while simultaneously introducing augmented capabilities. Empirical evaluations across a variety of benchmarks underscore InfiMM{'}s remarkable capability in multimodal understanding. The code can be found at: https://anonymous.4open.science/r/infimm-zephyr-F60C/.",
}
models
3
![](https://cdn-avatars.huggingface.co/v1/production/uploads/6566ad47dee7d2c57a9dd125/8plPO2Ig_G5Vy6NyvOrf9.png)
![](https://cdn-avatars.huggingface.co/v1/production/uploads/6566ad47dee7d2c57a9dd125/8plPO2Ig_G5Vy6NyvOrf9.png)
Infi-MM/infimm-vicuna13b
![](https://cdn-avatars.huggingface.co/v1/production/uploads/6566ad47dee7d2c57a9dd125/8plPO2Ig_G5Vy6NyvOrf9.png)