Nonsyndromic orofacial clefts (OFCs) are a common birth defect and are phenotypically heterogenous in the structure affected by the cleft - cleft lip (CL) and cleft lip and palate (CLP) - as well as other features, such as the severity of the cleft. Here, we focus on bilateral and unilateral clefts as one dimension of OFC severity, because the genetic architecture of these subtypes is not well understood. We tested for subtype-specific genetic associations in 44 bilateral CL (BCL) cases, 434 unilateral CL (UCL) cases, 530 bilateral CLP cases (BCLP), 1123 unilateral CLP (UCLP) cases, and unrelated controls (N = 1626), using a mixed-model approach. While no novel loci were found, the genetic architecture of UCL was distinct compared to BCL, with 44.03% of suggestive loci having different effects between the two subtypes. To further understand the subtype-specific genetic risk factors, we performed a genome-wide scan for modifiers and found a significant modifier locus on 20p11 (p=7.53×10-9), 300kb downstream of PAX1, that associated with higher odds of BCL vs. UCL, and replicated in an independent cohort (p=0.0018) with no effect in BCLP (p>0.05). We further found that this locus was associated with normal human nasal shape. Taken together, these results suggest bilateral and unilateral clefts may have different genetic architectures. Moreover, our results suggest BCL, the rarest form of OFC, may be genetically distinct from the other OFC subtypes. This expands our understanding of modifiers for OFC subtypes and further elucidates the genetic mechanisms behind the phenotypic heterogeneity in OFCs.