Domestic dog breeds display significant diversity in both body mass and skeletal size, resulting from intensive selective pressure during the formation and maintenance of modern breeds. While previous studies focused on the identification of alleles that contribute to small skeletal size, little is known about the underlying genetics controlling large size. We first performed a genome-wide association study (GWAS) using the Illumina Canine HD 170,000 single nucleotide polymorphism (SNP) array which compared 165 large-breed dogs from 19 breeds (defined as having a Standard Breed Weight (SBW) >41 kg [90 lb]) to 690 dogs from 69 small breeds (SBW ≤41 kg). We identified two loci on the canine X chromosome that were strongly associated with large body size at 82-84 megabases (Mb) and 101-104 Mb. Analyses of whole genome sequencing (WGS) data from 163 dogs revealed two indels in the Insulin Receptor Substrate 4 (IRS4) gene at 82.2 Mb and two additional mutations, one SNP and one deletion of a single codon, in Immunoglobulin Superfamily member 1 gene (IGSF1) at 102.3 Mb. IRS4 and IGSF1 are members of the GH/IGF1 and thyroid pathways whose roles include determination of body size. We also found one highly associated SNP in the 5'UTR of Acyl-CoA Synthetase Long-chain family member 4 (ACSL4) at 82.9 Mb, a gene which controls the traits of muscling and back fat thickness. We show by analysis of sequencing data from 26 wolves and 959 dogs representing 102 domestic dog breeds that skeletal size and body mass in large dog breeds are strongly associated with variants within IRS4, ACSL4 and IGSF1.