Faithfully storing an unknown quantum light state is essential to advanced quantum communication and distributed quantum computation applications. The required quantum memory must have high fidelity to improve the performance of a quantum network. Here we report the reversible transfer of photonic polarization states into collective atomic excitation in a compact solid-state device. The quantum memory is based on an atomic frequency comb (AFC) in rare-earth ion-doped crystals. We obtain up to 0.999 process fidelity for the storage and retrieval process of single-photon-level coherent pulse. This reliable quantum memory is a crucial step toward quantum networks based on solid-state devices.