-
LoRA-Ensemble: Efficient Uncertainty Modelling for Self-attention Networks
Authors:
Michelle Halbheer,
Dominik J. Mühlematter,
Alexander Becker,
Dominik Narnhofer,
Helge Aasen,
Konrad Schindler,
Mehmet Ozgur Turkoglu
Abstract:
Numerous crucial tasks in real-world decision-making rely on machine learning algorithms with calibrated uncertainty estimates. However, modern methods often yield overconfident and uncalibrated predictions. Various approaches involve training an ensemble of separate models to quantify the uncertainty related to the model itself, known as epistemic uncertainty. In an explicit implementation, the e…
▽ More
Numerous crucial tasks in real-world decision-making rely on machine learning algorithms with calibrated uncertainty estimates. However, modern methods often yield overconfident and uncalibrated predictions. Various approaches involve training an ensemble of separate models to quantify the uncertainty related to the model itself, known as epistemic uncertainty. In an explicit implementation, the ensemble approach has high computational cost and high memory requirements. This particular challenge is evident in state-of-the-art neural networks such as transformers, where even a single network is already demanding in terms of compute and memory. Consequently, efforts are made to emulate the ensemble model without actually instantiating separate ensemble members, referred to as implicit ensembling. We introduce LoRA-Ensemble, a parameter-efficient deep ensemble method for self-attention networks, which is based on Low-Rank Adaptation (LoRA). Initially developed for efficient LLM fine-tuning, we extend LoRA to an implicit ensembling approach. By employing a single pre-trained self-attention network with weights shared across all members, we train member-specific low-rank matrices for the attention projections. Our method exhibits superior calibration compared to explicit ensembles and achieves similar or better accuracy across various prediction tasks and datasets.
△ Less
Submitted 23 May, 2024;
originally announced May 2024.
-
Global Wheat Head Dataset 2021: more diversity to improve the benchmarking of wheat head localization methods
Authors:
Etienne David,
Mario Serouart,
Daniel Smith,
Simon Madec,
Kaaviya Velumani,
Shouyang Liu,
Xu Wang,
Francisco Pinto Espinosa,
Shahameh Shafiee,
Izzat S. A. Tahir,
Hisashi Tsujimoto,
Shuhei Nasuda,
Bangyou Zheng,
Norbert Kichgessner,
Helge Aasen,
Andreas Hund,
Pouria Sadhegi-Tehran,
Koichi Nagasawa,
Goro Ishikawa,
Sébastien Dandrifosse,
Alexis Carlier,
Benoit Mercatoris,
Ken Kuroki,
Haozhou Wang,
Masanori Ishii
, et al. (10 additional authors not shown)
Abstract:
The Global Wheat Head Detection (GWHD) dataset was created in 2020 and has assembled 193,634 labelled wheat heads from 4,700 RGB images acquired from various acquisition platforms and 7 countries/institutions. With an associated competition hosted in Kaggle, GWHD has successfully attracted attention from both the computer vision and agricultural science communities. From this first experience in 2…
▽ More
The Global Wheat Head Detection (GWHD) dataset was created in 2020 and has assembled 193,634 labelled wheat heads from 4,700 RGB images acquired from various acquisition platforms and 7 countries/institutions. With an associated competition hosted in Kaggle, GWHD has successfully attracted attention from both the computer vision and agricultural science communities. From this first experience in 2020, a few avenues for improvements have been identified, especially from the perspective of data size, head diversity and label reliability. To address these issues, the 2020 dataset has been reexamined, relabeled, and augmented by adding 1,722 images from 5 additional countries, allowing for 81,553 additional wheat heads to be added. We now release a new version of the Global Wheat Head Detection (GWHD) dataset in 2021, which is bigger, more diverse, and less noisy than the 2020 version. The GWHD 2021 is now publicly available at http://www.global-wheat.com/ and a new data challenge has been organized on AIcrowd to make use of this updated dataset.
△ Less
Submitted 3 June, 2021; v1 submitted 17 May, 2021;
originally announced May 2021.
-
Global Wheat Head Detection (GWHD) dataset: a large and diverse dataset of high resolution RGB labelled images to develop and benchmark wheat head detection methods
Authors:
E. David,
S. Madec,
P. Sadeghi-Tehran,
H. Aasen,
B. Zheng,
S. Liu,
N. Kirchgessner,
G. Ishikawa,
K. Nagasawa,
M. A. Badhon,
C. Pozniak,
B. de Solan,
A. Hund,
S. C. Chapman,
F. Baret,
I. Stavness,
W. Guo
Abstract:
Detection of wheat heads is an important task allowing to estimate pertinent traits including head population density and head characteristics such as sanitary state, size, maturity stage and the presence of awns. Several studies developed methods for wheat head detection from high-resolution RGB imagery. They are based on computer vision and machine learning and are generally calibrated and valid…
▽ More
Detection of wheat heads is an important task allowing to estimate pertinent traits including head population density and head characteristics such as sanitary state, size, maturity stage and the presence of awns. Several studies developed methods for wheat head detection from high-resolution RGB imagery. They are based on computer vision and machine learning and are generally calibrated and validated on limited datasets. However, variability in observational conditions, genotypic differences, development stages, head orientation represents a challenge in computer vision. Further, possible blurring due to motion or wind and overlap between heads for dense populations make this task even more complex. Through a joint international collaborative effort, we have built a large, diverse and well-labelled dataset, the Global Wheat Head detection (GWHD) dataset. It contains 4,700 high-resolution RGB images and 190,000 labelled wheat heads collected from several countries around the world at different growth stages with a wide range of genotypes. Guidelines for image acquisition, associating minimum metadata to respect FAIR principles and consistent head labelling methods are proposed when developing new head detection datasets. The GWHD is publicly available at http://www.global-wheat.com/ and aimed at developing and benchmarking methods for wheat head detection.
△ Less
Submitted 30 June, 2020; v1 submitted 25 April, 2020;
originally announced May 2020.
