Mobile phone applications (MPAs) for substance use disorder (SUD) treatment are increasingly used by patients. Although pilot studies have shown promising results, multiple previous systematic reviews noted insufficient evidence for MPA use in SUD treatment-many of the previously published reviews evaluated different trials. Subsequently, we aimed to conduct an umbrella review of previously published reviews investigating the efficacy of MPAs for SUD treatment, excluding nicotine/tobacco because umbrella reviews have been done in this population and the nicotine/tobacco MPA approach often differs from SUD-focused MPAs. No previous reviews have included a statistical meta-analysis of clinical trials to quantify an estimated overall effect. Seven reviews met inclusion criteria, and 17 unique studies with available data were taken from those reviews for the meta-analysis. Overall, reviews reported a lack of evidence for recommending MPAs for SUD treatment. However, MPA-delivered recovery support services, cognitive behavioral therapy, and contingency management were identified across multiple reviews as having promising evidence for SUD treatment. Hedges g effect size for an MPA reduction in substance use-related outcomes relative to the control arm was insignificant (0.137; 95% CI, -0.056 to 0.330; P=.16). In subgroup analysis, contingency management (1.29; 95% CI, 1.088-1.482; τ 2=0; k=2) and cognitive behavioral therapy (0.02; 95% CI, 0.001-0.030; τ 2=0; k=2) were significant. Although contingency management's effect was large, both trials were small (samples of 40 and 30). This review includes an adapted framework for the American Psychiatric Association's MPA guidelines that clinicians can implement to review MPAs critically with patients.