Minority groups are more prone to worsen their personal and social vulnerabilities during the COVID-19 pandemic. This study aimed to identify factors associated with the highest COVID-19 vulnerability in the Brazilian sexual and gender minorities. This is a cross-sectional study based on 826 respondents of the Brazilian LGBT+ Health Survey, conducted online from August to November 2020. The COVID-19 vulnerability was based on a previous vulnerability index created by an LGBT+ institution, which comprises three dimensions (income, COVID-19 exposure, and health). The outcome was the highest score quartile. Statistical analysis was based on logistic regression models. The COVID-19 vulnerability was higher in heterosexual and other scarce sexual orientations (OR = 2.34; 95%CI: 1.01-9.20, vs. homosexual), cisgender men (OR = 3.52; 95%CI: 1.35-4.44, vs. cisgender women), and those aged ≥ 50 years (OR = 3.74; 95%CI: 1.24-11.25, vs. 18-29 years old). A negative association was found with complete graduate education (OR = 0.06; 95%CI: 0.02-0.22, vs. complete high school), being white (OR = 0.44; 95%CI: 0.23-0.83), and proper facemask use (OR = 0.31; 95%CI: 0.13-0.76). Except for proper facemask use, factors associated with higher COVID-19 vulnerability are structural determinate and suggest overlapping vulnerabilities, as described by the syndemic model. It guides strategies to deal with the pandemic, which includes a joint approach to the common epidemic that affects sexual and gender minorities, broadening the intersectoral approach to decrease inequalities.