The formation process of intermediate-mass black holes (IMBHs), defined as those between 100 and 105 solar masses (M⊙), is debated. One potential origin is the growth of less-massive black holes merging with stars and compact objects within globular clusters (GCs). However, previous simulations have indicated that this process only produces IMBHs under 500 M⊙ before gravitational wave recoil ejects them from the GC. We performed star-by-star simulations of GC formation, finding that high-density star formation in a GC's parent giant molecular cloud can produce sufficient mergers of massive stars to overcome that mass threshold. We conclude that GCs can form with IMBHs more than 103 M⊙, which is sufficiently massive to be retained within the GC even with the expected gravitational wave recoil.