Destructive agents, who opt out of the game and indiscriminately harm others, paradoxically foster cooperation, representing an intriguing variant of the voluntary participation strategy. Yet, their impact on cooperation remains inadequately understood, particularly in the context of pairwise social dilemma games and in comparison to their counterparts, constructive agents, who opt out of the game but indiscriminately benefit others. Furthermore, little is known about the combined effects of both agent types on cooperation dynamics. Using replicator dynamics in infinite and well-mixed populations, we find that contrary to their role in facilitating cooperation in multiplayer games, destructive agents fail to encourage cooperation in pairwise social dilemmas. Instead, they replace defection in the prisoners' dilemma and stag-hunt games. Similarly, in the chicken game, they can destabilize or replace the mixed equilibrium of cooperation and defection, undermining cooperation in the harmony (trivial) game. Conversely, constructive agents, when their payoffs exceed their contributions to opponents, can exhibit effects similar to destructive agents. However, if their payoffs are lower, while they destabilize defection in prisoners' dilemma and stag-hunt games, they do not disrupt the cooperation equilibrium in harmony games and have a negligible impact on the coexistence of cooperation in chicken games. The combination of destructive and constructive agents does not facilitate cooperation, but instead generates complex evolutionary dynamics, including bistable, tristable, and quadstable states, with outcomes contingent on their relative payoffs and game types. These results, taken together, enhance our understanding of the impact of the voluntary participation mechanism on cooperation, contributing to a more comprehensive understanding of its influence.