We describe the construction of an apparatus designed to realize a hybrid quantum system comprised of a cryogenically cooled mechanical oscillator and ultra-cold 87Rb atoms coupled via light. The outstanding feature of our instrument is an in situ adjustable asymmetric all-fiber membrane-in-the-middle cavity located inside an ultra-high vacuum dilution refrigerator based cryostat. We show that Bose-Einstein condensates of N=2×106 atoms can be produced in less than 20 s and demonstrate a single photon optomechanical coupling strength of g0=2π×9 kHz employing a high-stress Si3N4 membrane with a mechanical quality factor Qm>107 at a cavity setup temperature of TMiM = 480 mK.