Cobalt phosphide (CoP) is a potential alternative to Li-ion battery (LIB) anodes due to its high specific capacity. However, there remain challenges, including low rate capability and rapid capacity degradation, because of its structural pulverization and poor electrical conductivity. Here, we demonstrate an effective strategy to enhance CoP-based anodes by developing a CoP/graphene nanocomposite. Such a nanocomposite can be achieved by embedding nanostructured CoP in a reduced graphene oxide (rGO) nanosheet network through a versatile method including the low-temperature formation of metal oxide nanoparticles, freeze-drying, and a subsequent phosphidation process. Benefiting from its favorable nanoarchitecture, the CoP/rGO nanocomposite is found to possess enhanced conductivity, porosity and structural stability. As a result, the nanocomposite shows a high specific capacity up to 1154 mA h g-1 at a current density of 100 mA g-1 and a remarkable rate capability (840 mA h g-1 at 2 A g-1). Moreover, a high capacity of 808 mA h g-1 is achieved even after 2000 cycles. These promising features indicate that our strategy could open the door to the further applications of CoP-based anodes in LIBs.