Learning-based Power Control for Secure Covert Semantic Communication

As a new communication paradigm, semantic communication (SemCom) focuses on transferring task-relevant information semantically, instead of on accurate bit-level transmission [1]. With the lightening of semantic models, this emerging form of communication presents unprecedented opportunities for network edge devices. However, research on SemCom security is still in its infancy. Traditional communication security technologies cannot be directly applied to SemCom [1]. Moreover, highly complex encryption mechanisms impose enormous pressures on low-power network edge devices [2]. With the development of covert communication and SemCom, covert SemCom has emerged as a novel approach to tackle the inherent security challenges in SemCom. In covert SemCom, the jammer transmits noise signals to conceal the transmitter’s activity, making potential wardens even unable to confirm the occurrence of communication activities [3]. Therefore, the communication process is hidden. And even if the communication is partially sniffed, since semantic-level encoding is adopted [4], the warden may not understand the true meaning of the sniffed information, thereby enhancing the concealment and security of communication under limited energy conditions.

However, a significant challenge for covert SemCom lies in improving concealment while maximizing the performance of SemCom under limited energy conditions. Current research on covert communication primarily focuses on traditional communication paradigms [5, 6, 7, 8]. In addition, research in the domain of SemCom security is still at a nascent stage [4, 9]. To the best of our knowledge, covert SemCom has merely been mentioned in [10], which is tailored to specific image question-answering tasks, lacking generality. Thus, covert SemCom necessitates more comprehensive investigations.

To address the challenge mentioned above, we propose a novel general covert SemCom framework to tackle the security challenges of SemCom under limited energy conditions. Specifically, we consider the power control problem in covert SemCom to ensure semantic communication quality under overall energy limitations. Then, we propose a learning-based approach based on the Soft Actor-Critic (SAC) algorithm [11] to optimize power control within SemCom systems. This approach not only safeguards the effectiveness of covert communication but also accurately decodes the semantic information. Our contributions are summarized as follows:

• $\bullet$

  A general covert semantic system framework: The proposed framework seamlessly integrates covert communication principles, enabling its application across diverse data modalities including text, image, and audio. This ensures its versatility and practicality in the context of multimodal data processing. Irrespective of the data type involved, our framework facilitates efficient and concealed semantic transmission.

• $\bullet$

  Power control for covert semantic communication: We consider power control for the covert SemCom scenario composed of a transmitter, a friendly jammer, a receiver, a warden, and a power regulator. Considering energy constraints and the aim of maximizing the performance of covert SemCom, we strive to optimize the power between the transmitter and the jammer, thus enhancing covert SemCom performance under limited energy conditions.

• $\bullet$

  Learning-based approach for covert semantic communication power control: We propose a SAC-based approach to power control in covert semantic communication, which not only safeguards the efficiency of covert communication but also accomplishes high-quality semantic decoding. Numerical results demonstrate that the proposed approach outperforms other Deep reinforcement learning algorithms.

We elaborate on the utilization of the SAC algorithm for optimal power control. The algorithm provides a power control scheme, typically denoted as $\{P_{t},P_{j}\}$, as seen in (4a). The reward of the agent (i.e., the power regulator) is determined by the actions executed pertaining to a particular state of the environment. Here are the details of the SAC formulation:

• •

  State space: The states, denoted as $\mathbf{s}$, are construed as a vector of measurements that encapsulate the conditions of the environment. In alignment with our discussion in Section II-B, herein, the environmental vector refers to the comprehensive set of all variables influencing the optimal power control scheme, given by

  	$\displaystyle\mathbf{s}=\{D_{\text{tw}},D_{\text{tr}},D_{\text{jw}},D_{\text{jr}},\alpha_{\text{tw}},\alpha_{\text{tr}},\alpha_{\text{jw}},\alpha_{\text{jr}},$		(5)
  	$\displaystyle\zeta_{\text{th}},h_{\text{tw}},h_{\text{tr}},h_{\text{jw}},h_{\text{jr}},\mathcal{K}^{2},\lambda,\tau\}.$

• •

  Action space: Based on the environment state, the agent initiates an action, denoted as $\mathbf{a}$. The action space represents the control actions taken by the power regulator, structured with several control variables. The proposed scheme uses transmit power $P_{t}$ and jamming power $P_{j}$ as actions of the power regulator, given by

  $\displaystyle\mathbf{a}=\{P_{t},P_{j}\}.$

  (6)

• •

  Immediate reward: In optimal power control, the goal is to maximize the BLEU score while minimizing the total energy consumption given by $\eta_{t}\times P_{t}$, $\eta_{j}\times P_{j}$, and ensuring the communications remain the covert communication constraint. To speed up the convergence, we consider the above two constraints in designing the reward functions, which is expressed as

  $\displaystyle R{(t)}=\begin{cases}\text{BLEU}{(t)},&\text{\noindent\hbox{}{\hbox{\hbox{\begin{tabular}[c]{@{}l@{}}if (\ref{eq:opta}), (\ref{eq:optb}), and (\ref{eq:optc}) are met\end{tabular}}}}\hfill\hbox{}},\\ \Psi,&\text{otherwise,}\end{cases}$

  (7)

  where $R{(t)}$ is the reward at time $t$, $\text{BLEU}{(t)}$ represents the degree of semantic retention and reconstruction quality of the source data and reconstructed data at time $t$, and $\Psi$ represents the penalty factor, which varies with the scale of the environment.

• •

  Value function: SAC meticulously optimizes a stochastic policy, maximizing both long-term entropy and expected lifetime rewards [11]. Concomitantly, it learns a policy $\pi_{\tau}(\mathbf{a}|\mathbf{s})$ and two Q-functions, $Q_{\theta_{1}}$ and $Q_{\theta_{2}}$, applying the minimum of these two Q-values to construct the targets in Bellman error functions, which is given by

  $$y(r,\mathbf{s},d)=\gamma(1-d)\big{(}\min_{i=1,2}Q_{\theta_{i}}(\mathbf{s},\mathbf{a})-\alpha\log\pi_{\tau}(\mathbf{a}|\mathbf{s})\big{)}+r,$$

  (8)

  where $d$ is indicative of the done signal, $\alpha$ serves as a coefficient that manages trade-offs, $\gamma$ is the discount factor, and $r$ denotes the reward. In every state, the policy executes actions to maximize the expected future return along with the expected future entropy, i.e., optimizing the state value function $V^{\pi}(\mathbf{s})$, which is given by

  $\displaystyle V^{\pi}(\mathbf{s})=\mathbb{E}_{\mathbf{a}\sim\pi}\left[Q^{\pi_{\tau}}(\mathbf{s},\mathbf{a})-\alpha\log\pi_{\tau}(\mathbf{a}|\mathbf{s})\right].$

  (9)

• •

  SAC algorithm design: We consider the power regulator as a SAC agent and a stochastic policy $\pi(\mathbf{a}_{n}|\mathbf{s}_{n})$ is defined to map from states to a probability distribution over actions. At each training iteration, the power regulator observes the state $\mathbf{s}_{n}$ and executes an action $\mathbf{a}_{n}$ sampled from the current policy $\pi$. Then, the environment turns into the next state $\mathbf{s}_{n+1}$ and sends back the reward $r_{n}$ to the power regulator. The tuple $\mathbf{e}_{n}$ is stored as experience in a data set $\mathcal{D}$ called replay buffer. By sampling from $\mathcal{D}$ and updating the policy periodically according to a learning algorithm, the power regulator will finally find an optimal policy $\pi^{*}$ that maximizes the long-term reward (10). The specific learning process can be seen from the Algorithm 1.

  $\displaystyle R_{\pi}=\mathbb{E}_{\pi}\left[\sum_{n=0}^{\infty}\gamma^{n}r(\mathbf{s}_{n},\mathbf{a}_{n})\right].$

  (10)

The effects of increasing transmit power on various parameters, such as the covert rate, defined as the data rate attainable during covert communications, DEP, and BEP, are illustrated. These effects are observed when the jamming power, $P_{j}$, is set to $45\>\rm{dBW}$ and $\tau$ is $50$. Within a Cartesian coordinate grid where the units are in meters, the transmitter, warden, receiver, jammer, and power regulator are positioned at coordinates $(0,0)$, $(0,100)$, $(100,0)$, $(100,100)$ and $(50,50)$, respectively. The path loss exponents are set to $\alpha_{tr}=1$, $\alpha_{tw}=1.2$, and both $\alpha_{jw}$ and $\alpha_{jr}$ are configured to $1.4$. As for small-scale channel fading attributes such as $h_{tw}$, $h_{tr}$ , $h_{jw}$, and $h_{jr}$, they are designed to conform with the $\alpha-\mu$ fading model, with the parameters $\alpha$ and $\mu$ being $2$ and $4$, respectively [16].

The adopted channel coding strategy is Binary Phase-shift Keying (BPSK). As demonstrated in Fig. 2, given the jamming power, the gradual increase in transmission power undoubtedly leads to a direct increase in the covert rate. However, as the transmission power gradually increases, the signal power received by the warden gradually reaches its detection range. This increases the probability of being monitored by the warden, i.e., the DEP of the warden gradually decreases. For the receiver, there are not only data signals from the transmitter in the environment but also noise signals from the friendly jammer. When the transmit power is smaller and the noise power is larger, even though the trained SemCom model can well extract the semantic information, the final recovery result will be very poor due to the high BEP. As shown in Fig. 2, when the transmission power is approximately $49.5$ $\rm{dBW}$, it can ensure covert communication while creating good channel conditions for semantic information reconstruction.

Figure 3 presents the performance analysis of the proposed SAC-based algorithm for optimal power control. We compare the proposed SAC-based algorithm with other algorithms: 1) PPO-based algorithm that the power regulator utilizes prioritizes optimizing control strategies based on data reconstruction performance; 2) GDM-based algorithm that the power regulator designs schemes by denoising noise to the initial Gaussian noise to recover or discover the optimal actions [17]. As shown in Fig. 3, the proposed SAC-based algorithm outperforms PPO and GDM algorithms. Specifically, the SAC-based algorithm converges to good results faster compared to PPO, while PPO achieves satisfactory results after approximately 3500 iterations, indicating that the proposed SAC-based can more effectively find the optimal scheme. The reason is that the SAC algorithm, by maximizing policy entropy, helps converge to the optimal solution more effectively, enabling it to reach better solutions in a shorter time compared to the PPO and GDM algorithms [11, 17].

Figure 4 shows the reconstruction process under varying BEPs. We can observe that when BEP is relatively small in Case 1, the large-scale models significantly outperform the other two. This is because the transmission power is higher than the noise, and under sufficiently good channel conditions, large-scale models can effectively extract and reconstruct semantic information. However, as BEP gradually increases from $0.11$ to $0.24$ in Case 2, it is clear that small-scale models can achieve better performance. This is due to the low transmission power or high interference power, resulting in a low signal-to-noise ratio. The small-scale models can alleviate the interference caused by noise more effectively compared with the large-scale models. Finally, when BEP exceeds $0.24$ in Case 3, the channel conditions have already become very poor and any type of model is unable to decode semantic information correctly. The reconstructed data always appear as noise, resulting in relatively low BLEU scores. This further emphasizes the importance of optimizing power control.

In this paper, we proposed a learning-based approach to optimal power control for the proposed covert SemCom, aimed at resolving the security dilemmas inherent in SemCom. Specifically, we introduced a power regulator and a friendly jammer to achieve covert communication within the context of SemCom. To address the increased energy usage introduced by covert SemCom, we formulated power control as an optimization problem and utilized the SAC algorithm to optimize power control between the transmitter and the friendly jammer, which strikes a balance between the effectiveness of covert transmission and SemCom. Numerical results have demonstrated the effectiveness of the proposed optimization strategy within the covert SemCom framework. For future work, we will explore ways to integrate the concept of the Mixture of Experts (MoE) system into our covert SemCom framework, ensuring reliable communication across varied network conditions.