Self-assembly of chemically modified graphenes (CMGs), including graphene oxide (GO), reduced graphene oxide (RGO) and their derivatives, has emerged as one of the most appealing strategies to develop unprecedented graphene-based functional materials. With the assistance of various non-covalent forces such as hydrogen bonding, ionic, amphiphilic and π-π interactions, CMGs decorated with multiple functional groups are favorable for assembly with different organic and inorganic components which can result in hierarchical composites possessing unique structures and functions. In this review, we will summarize the state-of-the-art self-assembly strategies that have been established to construct CMG based nanomaterials, including nanoparticles, nanospheres, nanofibers, nanorods, nanosheets, and macroscopic thin films, fibers and porous networks. The driving forces involved in the self-assembly process will be elucidated in the context. Further, we will also highlight several representative examples of applications regarding the self-assembled CMG based materials.