Predicting protein-protein interaction (PPI) binding affinities in unseen protein complex clusters is essential for elucidating complex protein interactions and for the targeted screening of peptide- or protein-based drugs. We introduce MCGLPPI++, a meta-learning framework designed to improve the adaptability of pretrained geometric models in such scenarios. To effectively boost the meta-learning optimization by injecting prior intersample distribution knowledge, three specially designed training sample cluster splitting patterns based on protein interaction interfaces are introduced. Additionally, MCGLPPI++ is equipped with an independent energy component which explicitly models interface nonbonded interaction energies closely related to the strengths of PPIs. To validate our approach, we curate a new data set featuring a challenging test cluster of T-cell receptors binding to antigenic peptide-MHC molecules (TCR-pMHC). Experimental results show that geometric models enhanced by the MCGLPPI++ framework achieve significantly more robust binding affinity predictions after fine-tuning on a few samples from this novel cluster compared to their vanilla counterparts, which demonstrates the effectiveness of the framework.