We demonstrate a source for correlated pairs of atoms characterized by two opposite momenta and two spatial modes forming a Bell state only involving external degrees of freedom. We characterize the state of the emitted atom beams by observing strong number squeezing up to -10 dB in the correlated two-particle modes of emission. We furthermore demonstrate genuine two-particle interference in the normalized second-order correlation function g^{(2)} relative to the emitted atoms.