Distributional Black-Box Model Inversion Attack with Multi-Agent Reinforcement Learning

This work primarily examines black-box MI attacks, where the attacker can neither access the private data $D_{priv}$ of the black-box target network $T$ nor obtain any knowledge about the model’s structure, hyper-parameters, etc. The black-box target model $T$ is trained to recognize $k$ different identities using a private dataset $D_{priv}$. The model $T$ produces a probability distribution according to the following way:

where $[0,1]^{k}$ represents the probabilities of $x$ being classified into each of the $k$ identities.

We can only obtain the corresponding $T(x)$ by inputting an image $x$ into the target model for discrimination in the black-box model without obtaining any intermediate parameters from any of the models. The attacker aims to obtain sensitive data $x$ associated with a specific label $y$ from the target network. We chose the face recognition classifier model $T$ as the attack target to make our attack more realistic. This model identifies individuals’ identities in images and assigns the corresponding labels. Thus, the private facial images of any specific identity are constructed by utilizing the soft and hard labels provided by the black-box model.

It is essential to satisfy the following conditions to expose the privacy of the target $y$ by reconstructing data $x$: 1) The probability of the target label $y$ in the prediction probability distribution of model $T$ on input $x$ must be maximized, i.e., $argmax_{y}(T(x)_{y})=y$. 2) The confidence level of the label $y$ should be as large as possible, i.e., maximizing $T(x)_{y}$.

In GANs, the variation of latent code in the latent space is continuous. Hence, searching for latent code can be regarded as a Markov decision process (MDP). However, searching for latent distribution cannot be considered an MDP. Firstly, latent distribution has a more complex state space than latent code, and the parameters constituting the distribution, such as mean $\mu$ and variance $\sigma$, entail more uncertainty and interaction. Therefore, searching for latent distribution should be viewed as a stochastic game.

When dealing with stochastic game problems, multi-agent reinforcement learning is often a good choice as it can effectively address interactions and competitions. Therefore, we select two agents to optimize the mean $\mu$ and variance $\sigma$ constituting the latent distribution, respectively. In the context of distributional MI attacks, these two agents aim to optimize the appropriate latent distribution for selecting appropriate latent code to reconstruct more privacy-preserving images. Even so, we cannot classify this task as entirely cooperative. A certain degree of competition exists between $\mu$ and $\sigma$. Maintaining this competitive dynamic enables them to enhance their performance while continuously striving for global optimality. This competitive relationship fosters flexibility in the optimization process, ultimately leading to improved MI attack performance. Therefore, we choose the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) [19] as the MARL agent for searching appropriate latent distribution.

DBB-MI consists of three steps. Firstly, we train a GAN, where the GAN $G$ is initially trained on the public dataset $D_{pub}$. It’s important to note that the public dataset $D_{pub}$ does not overlap with the private dataset $D_{priv}$ used to train the target black-box model $T$. We neither need to select the images carefully nor to use the target model $T$ for additional labelling of images in $D_{pub}$. Next, we optimize the initial random latent distribution to approximate the real latent distribution. Finally, we sample the latent code from the optimized high-dimensional latent distribution and input it into the GAN $G$ to reconstruct private data. The overall structure of DBB-MI is exhibited in Fig.2.

The MADDPG [19] agent has two fundamental components: the actor and critic networks, as depicted in Fig.3. The actor network determines the actions to be executed by the agent based on the information observed by the agent as well as the current state of the system. The critic network is responsible for judging the value of actions and providing feedback (i.e., a reward signal) to the actor. This process enables the actor network to update its policies. Through iterations of the above operations, the agent gradually learns to optimize parameters $\mu$ and $\sigma$ to minimize the discrepancy between the optimized and real latent distribution.

The actor network aggregates all the data the agent has observed, including the environment’s current state, other agents’ observations, and other pertinent information. Based on this information, the actor network makes decisions regarding the agent’s actions to optimize the latent distribution parameters $\mu$ and $\sigma$. We independently model the latent distribution of each dimension of the latent code and then sample each dimension independently from these latent distributions. Fig.2 shows that $\mu$ and $\sigma$ are sampled from the standard normal distribution to form the initial random distribution $N(\mu,\sigma)$. In addition, actions $action_{\mu}$ and $action_{\sigma}$ are selected by $Actor_{\mu}$ and $Actor_{\sigma}$ according to the initial parameters.

The following technique is employed to update actions:

Previous research [25, 28] has demonstrated that exploring latent space impacts the diversity and accuracy of reconstructed images. So, we introduce a parameter $\alpha$ to balance accuracy and diversity. A small $\alpha$ value is employed in the early stages of searching to encourage the agent to optimize the distribution, broadening the search scope. As the training process advances, the latent distribution optimized by agents eventually approaches the real latent space distribution. The $\alpha$ is gradually increased to mitigate the diversity of the generated images. The optimization procedure enables agents to refine the latent distribution further to generate a latent distribution closely related to natural images.

The critic network evaluates the value and utility of the actions by measuring their consistency with the target task. When optimizing the latent distribution, the critic network provides feedback or rewards to motivate agents to take actions to enable it toward the real latent space. Actions that steer the latent distribution toward the real latent space will be rewarded more significantly; otherwise, only a lower or no reward will be given.

As evidenced in Fig.3, the critic network considers the effect of actions on the state of the environment. It is updated by incorporating environmental feedback, actual rewards, and estimated action rewards. These help to improve the estimation of rewards by the critic networks. The agents move closer to the true latent distribution by optimizing actor and critic networks. When the current distribution is closer to the real latent distribution, the images generated from that distribution will have higher confidence in the target network $T$. Therefore, the reward can be calculated as follows:

where $r_{t+1}$ denotes the score of images generated from the latent space of the new distribution after performing actions, and $r_{a}$ is the reward that agents receive after performing actions.

In optimizing the latent distribution, it is necessary to compute rewards individually for each action based on its effectiveness and impact on the target. The reward calculation way helps improve the precision of dynamic adjustments in the optimization process, as formulated in the following:

We also introduce the penalty factor $r_{c}$ to penalize instances where the generated image is irrelevant to the target category. This could help agents perform actions to improve the quality of the generated image while reducing interference from non-target categories. The penalty term can assist agents in optimizing the distribution so that the reconstructed images are closely related to the target category, yielding superior-quality images. The penalty factor $r_{c}$ is defined below.

The threshold $\varepsilon$ is introduced to prevent agents from obtaining additional rewards. When the negative log probability ($-p_{l}$) of the target label of the generated image exceeds the specific threshold, an additional penalty is imposed. This penalty factor ensures that the generated image is more relevant to the target category and that images are distinguishable enough to avoid confusion with non-target categories.

To sum up, we can calculate rewards for agent_μ and agent_σ as follows:

where w_n represents the weight of r_n.

As shown in Fig.4, the search is conducted on the high-dimensional latent distribution using MADDPG. Specifically, the $\mu=\{\mu_{1},\mu_{2},...,\mu_{n}\}$ and $\sigma=\{\sigma_{1},\sigma_{2},...,\sigma_{n}\}$ are sampled from an $n$-dimensional normal distribution. After that, they are paired together to form the initial $n$-dimensional high-dimensional latent distribution as follows:

The MADDPG algorithm, involving $Agent_{\mu}$ and $Agent_{\sigma}$, optimizes the initial high-dimensional latent distribution. $Agent_{\mu}$ and $Agent_{\sigma}$ select actions based on the current initial distribution, and these actions are formulated below.

The chosen actions facilitate the latent distribution $L$ towards the real latent distribution, resulting in a highly optimized latent distribution $L^{\prime}=N(\mu^{\prime},{\sigma^{\prime}}^{2})$. The latent code $z^{\prime}$ is ultimately extracted from $L^{\prime}$.

where $z^{\prime}_{i}$ obeys the distribution $N(\mu^{\prime}_{i},{\sigma^{\prime}_{i}}^{2})$.

The above optimization method of high-dimensional latent distribution can avoid the limitation caused by directly searching the latent distribution space. Through MARL methods, an effective search of the latent distribution can be achieved with only limited model outputs, without the need for any additional information about the model. Consequently, in a black-box setting, it broadens the search range in the latent distribution space, thereby enhancing the ability to identify the real latent space of the target and ultimately improving the accuracy of the reconstructed sensitive data.

Four distinct face datasets that represent a variety of situations are used to evaluate the effectiveness and breadth of DBB-MI. Additionally, we conducted experiments on the MNIST dataset to assess the applicability of our approach to other types of datasets.

• •

  CelebFaces Attributes Dataset (CelebA) [29]. It contains 202,599 photos of 10,177 different celebrities.

• •

  FaceScrub [30]. It includes 106,863 images of 530 individuals with an even gender distribution.

• •

  Pubfig83 [31]. It consists of 13,600 images of 83 individuals. These images were taken in regulated real-world environments with significant variations in lighting, expressions, and other attributes.

• •

  Flickr-Faces-HQ (FFHQ) [32]. It comprises 70,000 high-quality face images with significant age, expression, and ethnicity variations.

• •

  MNIST. It encompasses 70,000 handwritten digits (0 through 9), having different structures and features from facial datasets.

CelebA, FaceScrub, Pubfig83, and MNIST are divided into two parts: public and private datasets. The public dataset is utilized to train GAN, while the private dataset is employed to train the target classification model. It should be emphasized that there is no overlap of the same identities or images between public and private datasets. Thus, it can be assumed that the trained GAN does not directly contain any original private information. Additionally, since public and private datasets in the same dataset have similar statistical properties, the FFHQ is used as an independent extra dataset to evaluate the performance of MI attacks under various distribution conditions. It allows for a comprehensive assessment of MI attacks’ robustness and generalization capabilities.

Like previous studies [12, 11, 27, 25], we utilize three popular face recognition networks for evaluation; namely FaceNet64 [33], ResNet-152 [34], and VGG16 [35]. These networks are employed to assess the impact of MI attacks on models with different architectures. The generalization and robustness of DBB-MI are better evaluated using varied face recognition models.

We select some representative state-of-the-art white-box and black-box MI attacks as baselines for comparison. Specifically, we choose the Generative Model Inversion (GMI) attacks [11] and Knowledge-Enriched Distributional Model Inversion (KED-MI) [12] attacks as white-box MI attacks. GMI is the first MI attack for deep networks, while KED-MI is a distributional MI attack. Meanwhile, we employ the Reinforcement Learning-based Black-box Model Inversion (RLB-MI) attacks [25] and Model Inversion for deep learning Network (MIRROR) [24], representing the advanced black-box MI attack. We also select the Boundary-Repelling Model Inversion (BERP-MI) attacks [27], the only and most advanced label-only MI attack. These black-box MI attacks represent the current state-of-the-art (SOTA) in GAN-based MI attacks that directly search for latent code.

All models undergo identical dataset training, and the same evaluation models assess all experimental results to ensure fair comparisons. GMI, RLB-MI, MIRROR, and BERP-MI utilize the same GAN as DBB-MI. In addition, the GAN is trained for KED-MI using the specified requirements and an identical dataset to that of DBB-MI. It allows a fair and objective comparison between KED-MI and DBB-MI.

The same hyperparameters are used to train the GAN and target network like previous studies [12, 11, 27, 25]. For MADDPG, some important parameters are set as follows:

• •

  learning rate: 1e-3

• •

  discount factor: 0.99

• •

  target network update rate: 5×$10^{-3}$

• •

  experience replay buffer size: 1×$10^{6}$

• •

  batch size: 256

• •

  training episodes: 4×$10^{4}$

Like prior work [11], the effectiveness of MI attacks is evaluated using quantitative criteria, including attack accuracy (ACC) and K-nearest neighbor feature distance (KNN Dist). Furthermore, the Peak Signal-to-Noise Ratio (PSNR) is utilized to assess the resemblance between reconstructed and original images.

Attack Accuracy:This metric measures the probability of successfully reconstructing private data through an attack. The key with [12, 25] difference is that we only consider the attack successful when both the target model and the additional discriminative model agree that the generated image belongs to the target class. This approach enhances the accuracy of attack success rate assessment, ensuring that the generated images deceive the target model and exhibit high-quality facial features, reducing cases where noisy images are mistakenly identified as the target class.

KNN Dist: This metric measures the similarity of features between the generated reconstructed images and real private images. To calculate the KNN Dist, features are first extracted from the fully connected layer of the evaluation classifier for both the generated reconstructed images and the real private images. Then, their similarity in the feature space is assessed by calculating the L2 distance between these two sets of features.

PSNR: This metric measures the difference between two images. It evaluates the quality and similarity between the reconstructed and real private images by calculating their PSNR value. A higher PSNR value indicates less difference between the two images, implying a higher similarity between the reconstructed and real private images.

Table I shows the experimental results of our method and baselines under different target models. As seen in Table I, DBB-MI exhibits a notable superiority compared to state-of-the-art white-box and black-box MI attacks regarding ACC, KNN Dist, and PSNR. Using the target model VGG16 as an example, the ACC of DBB-MI improves 25.4% over KED-MI and 33.6% over RLB-MI. This is because DBB-MI fully explores the latent space by optimizing the latent distribution and obtaining more private data about the target. The experimental results in Table I demonstrate that DBB-MI is more effective in targeting different target models and poses more severe privacy leakage risks. This indicates that our method outperforms white-box distributional attacks and achieves SOTA black-box attack performance.

In addition, we also compare the images reconstructed by DBB-MI with those rebuilt by baselines. As displayed in Fig.5, the images recovered by DBB-MI are closer to the original ones compared to those recovered by baselines; they have similar details and colors as the original. It is attributed to the efficiency of DBB-MI in searching the latent space, allowing it to capture more private information. The above experimental results prove that DBB-MI outperforms the state-of-the-art white-box and black-box MI attacks for various target models in terms of multiple performance evaluation metrics and visualization.

Table II presents the experimental results of DBB-MI and baselines under different datasets. It can be observed from Table II that DBB-MI beats baselines in all performance evaluation metrics. Taking CelebA as a case study, DBB-MI’s ACC is 19.5% and 13.9% higher than KED-MI and RLB-MI, respectively.

The experimental outcomes obtained by all MI attacks on CelebA are superior to those on FaceScrub and Pubfig83. This is because CelebA contains more identity categories and data information, giving the trained model stronger classification capabilities and, therefore, more vulnerability to attacks. Additionally, DBB-MI outperforms all baselines on all datasets except FaceScrub. On FaceScrub, the ACC of DBB-MI is slightly lower than that of RLB-MI, but the PSNR and KNN Dist of DBB-MI are better than those of RLB-MI. Thus, DBB-MI outperforms RLB-MI in most performance evaluation metrics.

In previous experiments, we utilized a dataset with similar statistical properties and feature distributions as the dataset used to train the target model to train GAN. However, obtaining a dataset with similar distributions to the target dataset in real-world settings is challenging. Therefore, it is imperative to train GAN using an extra dataset.

We train GAN on the extra dataset, FFHQ. Meanwhile, we employ the FaceNet64 trained under CelebA, FaceScrub, and Pubfig83 as the target models and utilize the FaceNet trained on these datasets as the evaluation models. Table III presents the experimental results of DBB-MI and baselines across datasets. It is easy to see that DBB-MI has superior performance compared to baselines across most datasets. The reason for this can be linked to the comprehensive exploration of the latent space in DBB-MI, which has resulted in the acquisition of more private data about the target. In addition, DBB-MI has the highest ACC on CelebA, while its PSNR and KNN Dist are slightly worse than the best. This implies that DBB-MI still has room for performance improvement.

In summary, DBB-MI exhibits superior performance across diverse target models, datasets, and cross-dataset scenarios. It outperforms state-of-the-art white-box and black-box MI attacks regarding ACC, KNN Dist, PSNR, and visualization. These findings confirm the effectiveness of the latent distribution exploration in DBB-MI, which is vital for improving model security and privacy protection.

To demonstrate that our approach is practical for face datasets, we also evaluated it on the MNIST dataset. For the MNIST dataset, we attacked models of different depths, including ResNet-18, ResNet-34, ResNet-50, ResNet-101, ResNet-152, and ResNet-SIM. ResNet-SIM consists of two convolutional layers, one max-pooling layer, two residual blocks, and one fully connected layer, while the remaining network structures conform to [34]. We achieved a 100% attack success rate across different networks, and features of different digits could be reconstructed. Specific reconstruction results are shown in Fig 6.

Fig. 7a depicts the latent distribution before attacking, in which each point represents a specific dimension of the latent distribution. During an attack, the latent distribution of each dimension is optimized by the MADDPG to explore the latent space effectively. Fig.7b illustrates the latent distribution after attacking, in which each dimension of the latent distribution displays a distinct pattern. This implies that distinct privacy features are evident in each dimension of latent distribution. Moreover, it also proves the rationality of DBB-MI, i.e., optimizing each dimension of the latent distribution independently to enable the latent distribution to approach the true one.

We conducted 500 random samples from the optimized latent distribution to obtain reconstructed samples. We calculated the proportion of recreated samples that matched the target labels to measure the accuracy of the optimized distribution. The accuracy of the optimized distribution was assessed by calculating the fraction of rebuilt samples that corresponded to the target labels, which is referred to as distributional accuracy. 68% of all latent distributions have an accuracy exceeding 0.5, indicating that more than half of the randomly sampled samples can be deemed successful reconstructions. As displayed in Fig.8, the specific results indicate that the optimized latent distribution demonstrates strong performance in effectively exploring the target label information.

To depict the real attack accuracy, we performed 10,000 random samples for all optimized latent distributions. These samples were then tested for top-1 accuracy, which indicates successful reconstruction, as well as top-5 accuracy, which signifies the target label ranking among the top 5 out of all 1,000 classes. When the latent distribution accuracy exceeds 24%, our latent distribution’s top-5 accuracy exceeds 50%. The observed top-1 accuracy closely aligns with the accuracy of the tested distribution, suggesting that the optimized distribution demonstrates consistent performance in all reconstruction tasks. More details as shown in Fig.9.

For testing, we selected samples from 10,000 random samplings of optimized latent distributions with varying accuracies. We evaluated the reconstructed samples using the target model to determine their corresponding label confidences, as depicted in Fig.10. When the distributional accuracy reached 0.632, it was observed that 50.59% of the rebuilt samples achieved a confidence level of 0.5, while 36.10% exhibited a confidence level of 0.75. However, upon attaining a distributional accuracy of 0.962, it was seen that 90.27% of the samples demonstrated a confidence level of 0.5, 78.58% exhibited a confidence level of 0.75, and 58.40% demonstrated a confidence level beyond 0.9. Despite the restricted distributional accuracy of 0.580, a significant proportion of the samples, specifically 47.51%, surpassed a confidence level of 0.5. Although there is an improvement in the accuracy of reconstructed samples as the distributional accuracy improves, it is important to note that certain samples still exhibit very low confidence levels. This may be attributed to the inadequate training of the GAN model employed in our study.

This work primarily concerns improving the performance of MI attacks by optimizing the latent distribution in the high-latitude latent space. Therefore, it is necessary to investigate the effect of varying latent distribution dimensions on the MI attack’s ACC. Fig.11 displays the ACC obtained by DBB-MI, with variations in latent distribution dimensions. As shown in this figure, the ACC of DBB-MI rises gradually with the increase of latent distribution dimensions. This suggests that searching high-dimensional latent distributions can explore the latent space more comprehensively. Hence, MI attacks need to choose the appropriate latent distribution dimension.

To further investigate the factors influencing the performance of MI attacks, we assess the median number of iterations required for generating images to reach the specific test classification model’s complexity and the dataset’s size. Experimental findings were obtained by training various target models on CelebA. An MI attack has the highest search difficulty to target ResNet-152, the most complex target model. More iterations are required to achieve the test accuracy obtained on FaceNet64 and VGG16, as shown in Fig.12a. Fig.12b exhibits the experimental results obtained by utilizing a GAN trained on FFHQ to attack FaceNet64 trained on different datasets. An MI attack exhibits the lowest search difficulty and can quickly reach a specific test accuracy on PubFig83, while it needs a higher number of iterations on other datasets. Therefore, we can conclude that as the complexity of the model and the size of the dataset increase, the search difficulty of an MI attack increases, ultimately leading to a decrease in the attack performance.

DBB-MI heavily relies on the MADDPG to optimize the latent distribution of GAN and obtain more private information about the target. To verify the rationality of using MADDPG, IQL [17], as a form of fully competitive MARL, and VDN [15], as a form of fully cooperative MARL, are used to search for the latent distribution for model inversion. Meanwhile, the VGG16 model trained on CelebA is employed as the target network. Table IV lists the experimental results of MI attacks using various reinforcement learning agents. This table illustrates that the performance of MADDPG surpasses that of IQL and VDN. For example, MADDPG’s ACC is 150% higher than IQL and 85.7% higher than VDN, respectively. This is why DBB-MI utilizes MADDPG to optimize GAN’s high-dimensional latent space distribution. Furthermore, this also underscores that searching for suitable latent distributions from the latent space of GANs should be regarded as a semi-competitive, semi-cooperative form of MARL.

To further assess the performance difference between various agents in GAN-based MI attacks, we compare the reward changes during their training, as shown in Fig.13. As can be seen from the figure, both VDN [15], and IQN [17] exhibit consistently modest rewards, with fluctuations occurring around this baseline amount. In contrast, MADDPG [19] can achieve higher reward convergence, yielding more gratifying results. Therefore, MADDPG is more suitable for hidden space search in GAN-based MI attacks.

It is expected to find several images associated with the same label, depicting various stages or conditions of the same object. DBB-MI can reconstruct several diverse images for a given label, as displayed in Fig.14. We sample multiple latent codes from the finally optimized latent distribution to generate multiple images with different privacy attributes. Here, variations in facial expressions, hair, lighting circumstances, and other variables highlight diverse privacy features within the same label. The diverse image reconstruction capabilities of DBB-MI are very important for studying privacy protection. It exposes how an attacker can reconstruct different state information of a target, which also implies how to defend against this attack effectively.