In this Letter, we report the first measurement of the inelastic cross section for antideuteron-nucleus interactions at low particle momenta, covering a range of 0.3≤p<4 GeV/c. The measurement is carried out using p-Pb collisions at a center-of-mass energy per nucleon-nucleon pair of sqrt[s_{NN}]=5.02 TeV, recorded with the ALICE detector at the CERN LHC and utilizing the detector material as an absorber for antideuterons and antiprotons. The extracted raw primary antiparticle-to-particle ratios are compared to the results from detailed ALICE simulations based on the geant4 toolkit for the propagation of (anti)particles through the detector material. The analysis of the raw primary (anti)proton spectra serves as a benchmark for this study, since their hadronic interaction cross sections are well constrained experimentally. The first measurement of the inelastic cross section for antideuteron-nucleus interactions averaged over the ALICE detector material with atomic mass numbers ⟨A⟩=17.4 and 31.8 is obtained. The measured inelastic cross section points to a possible excess with respect to the Glauber model parametrization used in geant4 in the lowest momentum interval of 0.3≤p<0.47 GeV/c up to a factor 2.1. This result is relevant for the understanding of antimatter propagation and the contributions to antinuclei production from cosmic ray interactions within the interstellar medium. In addition, the momentum range covered by this measurement is of particular importance to evaluate signal predictions for indirect dark-matter searches.