High-entropy ceramics hold promise for application as thermal barrier coating materials. However, a key challenge in practical applications lies in the low fracture toughness compared to that of yttria-stabilized zirconia (YSZ). Herein, we designed (Hf,Zr,Ce,M)O2-δ-Al2O3 (M = Y, Ca, and Gd) ceramic composites by following a set of fundamental guidelines. First-principles calculations predicted that the inclusion of Al2O3 in compositions containing the other four binary oxides decreased the propensity for single high-entropy phase formation. Instead, it increased the potential for Al2O3 to form a second phase within the high-entropy ceramic matrix, compared to compositions without Al2O3. Ceramic composites consisting of the Al2O3 second phase in a high-entropy fluorite oxide (Hf,Zr,Ce,M)O2-δ matrix were synthesized in situ via conventional solid-state reactions from the five constituent binary oxides. Both the hardness and fracture toughness of the ceramic composites were enhanced due to toughening mechanisms from the discrete Al2O3 particles, microcracks, and crack deflections. Additionally, the ceramic composites exhibited coefficients of thermal expansion and thermal conductivities comparable with those of YSZ. Our findings demonstrated the potential of the high-entropy (Hf,Zr,Ce,M)O2-δ-Al2O3 ceramic composites for advanced thermal barrier coating materials and offered a possible approach to reinforce other high-entropy oxide-based ceramic systems.