-
GINO-Q: Learning an Asymptotically Optimal Index Policy for Restless Multi-armed Bandits
Authors:
Gongpu Chen,
Soung Chang Liew,
Deniz Gunduz
Abstract:
The restless multi-armed bandit (RMAB) framework is a popular model with applications across a wide variety of fields. However, its solution is hindered by the exponentially growing state space (with respect to the number of arms) and the combinatorial action space, making traditional reinforcement learning methods infeasible for large-scale instances. In this paper, we propose GINO-Q, a three-tim…
▽ More
The restless multi-armed bandit (RMAB) framework is a popular model with applications across a wide variety of fields. However, its solution is hindered by the exponentially growing state space (with respect to the number of arms) and the combinatorial action space, making traditional reinforcement learning methods infeasible for large-scale instances. In this paper, we propose GINO-Q, a three-timescale stochastic approximation algorithm designed to learn an asymptotically optimal index policy for RMABs. GINO-Q mitigates the curse of dimensionality by decomposing the RMAB into a series of subproblems, each with the same dimension as a single arm, ensuring that complexity increases linearly with the number of arms. Unlike recently developed Whittle-index-based algorithms, GINO-Q does not require RMABs to be indexable, enhancing its flexibility and applicability. Our experimental results demonstrate that GINO-Q consistently learns near-optimal policies, even for non-indexable RMABs where Whittle-index-based algorithms perform poorly, and it converges significantly faster than existing baselines.
△ Less
Submitted 19 August, 2024;
originally announced August 2024.
-
A Versatile Pilot Design Scheme for FDD Systems Utilizing Gaussian Mixture Models
Authors:
Nurettin Turan,
Benedikt Böck,
Benedikt Fesl,
Michael Joham,
Deniz Gündüz,
Wolfgang Utschick
Abstract:
In this work, we propose a Gaussian mixture model (GMM)-based pilot design scheme for downlink (DL) channel estimation in single- and multi-user multiple-input multiple-output (MIMO) frequency division duplex (FDD) systems. In an initial offline phase, the GMM captures prior information during training, which is then utilized for pilot design. In the single-user case, the GMM is utilized to constr…
▽ More
In this work, we propose a Gaussian mixture model (GMM)-based pilot design scheme for downlink (DL) channel estimation in single- and multi-user multiple-input multiple-output (MIMO) frequency division duplex (FDD) systems. In an initial offline phase, the GMM captures prior information during training, which is then utilized for pilot design. In the single-user case, the GMM is utilized to construct a codebook of pilot matrices and, once shared with the mobile terminal (MT), can be employed to determine a feedback index at the MT. This index selects a pilot matrix from the constructed codebook, eliminating the need for online pilot optimization. We further establish a sum conditional mutual information (CMI)-based pilot optimization framework for multi-user MIMO (MU-MIMO) systems. Based on the established framework, we utilize the GMM for pilot matrix design in MU-MIMO systems. The analytic representation of the GMM enables the adaptation to any signal-to-noise ratio (SNR) level and pilot configuration without re-training. Additionally, an adaption to any number of MTs is facilitated. Extensive simulations demonstrate the superior performance of the proposed pilot design scheme compared to state-of-the-art approaches. The performance gains can be exploited, e.g., to deploy systems with fewer pilots.
△ Less
Submitted 7 August, 2024;
originally announced August 2024.
-
Delay Conditioned Generative Modelling of Resistive Drift in Memristors
Authors:
Waleed El-Geresy,
Christos Papavassiliou,
Deniz Gündüz
Abstract:
The modelling of memristive devices is an essential part of the development of novel in-memory computing systems. Models are needed to enable the accurate and efficient simulation of memristor device characteristics, for purposes of testing the performance of the devices or the feasibility of their use in future neuromorphic and in-memory computing architectures. The consideration of memristor non…
▽ More
The modelling of memristive devices is an essential part of the development of novel in-memory computing systems. Models are needed to enable the accurate and efficient simulation of memristor device characteristics, for purposes of testing the performance of the devices or the feasibility of their use in future neuromorphic and in-memory computing architectures. The consideration of memristor non-idealities is an essential part of any modelling approach. The nature of the deviation of memristive devices from their initial state, particularly at ambient temperature and in the absence of a stimulating voltage, is of key interest, as it dictates their reliability as information storage media - a property that is of importance for both traditional storage and neuromorphic applications. In this paper, we investigate the use of a generative modelling approach for the simulation of the delay and initial resistance-conditioned resistive drift distribution of memristive devices. We introduce a data normalisation scheme and a novel training technique to enable the generative model to be conditioned on the continuous inputs. The proposed generative modelling approach is suited for use in end-to-end training and device modelling scenarios, including learned data storage applications, due to its simulation efficiency and differentiability.
△ Less
Submitted 2 August, 2024;
originally announced August 2024.
-
Private Collaborative Edge Inference via Over-the-Air Computation
Authors:
Selim F. Yilmaz,
Burak Hasircioglu,
Li Qiao,
Deniz Gunduz
Abstract:
We consider collaborative inference at the wireless edge, where each client's model is trained independently on their local datasets. Clients are queried in parallel to make an accurate decision collaboratively. In addition to maximizing the inference accuracy, we also want to ensure the privacy of local models. To this end, we leverage the superposition property of the multiple access channel to…
▽ More
We consider collaborative inference at the wireless edge, where each client's model is trained independently on their local datasets. Clients are queried in parallel to make an accurate decision collaboratively. In addition to maximizing the inference accuracy, we also want to ensure the privacy of local models. To this end, we leverage the superposition property of the multiple access channel to implement bandwidth-efficient multi-user inference methods. Specifically, we propose different methods for ensemble and multi-view classification that exploit over-the-air computation. We show that these schemes perform better than their orthogonal counterparts with statistically significant differences while using fewer resources and providing privacy guarantees. We also provide experimental results verifying the benefits of the proposed over-the-air multi-user inference approach and perform an ablation study to demonstrate the effectiveness of our design choices. We share the source code of the framework publicly on Github to facilitate further research and reproducibility.
△ Less
Submitted 30 July, 2024;
originally announced July 2024.
-
Gaussian Channel Simulation with Rotated Dithered Quantization
Authors:
Szymon Kobus,
Lucas Theis,
Deniz Gündüz
Abstract:
Channel simulation involves generating a sample $Y$ from the conditional distribution $P_{Y|X}$, where $X$ is a remote realization sampled from $P_X$. This paper introduces a novel approach to approximate Gaussian channel simulation using dithered quantization. Our method concurrently simulates $n$ channels, reducing the upper bound on the excess information by half compared to one-dimensional met…
▽ More
Channel simulation involves generating a sample $Y$ from the conditional distribution $P_{Y|X}$, where $X$ is a remote realization sampled from $P_X$. This paper introduces a novel approach to approximate Gaussian channel simulation using dithered quantization. Our method concurrently simulates $n$ channels, reducing the upper bound on the excess information by half compared to one-dimensional methods. When used with higher-dimensional lattices, our approach achieves up to six times reduction on the upper bound. Furthermore, we demonstrate that the KL divergence between the distributions of the simulated and Gaussian channels decreases with the number of dimensions at a rate of $O(n^{-1})$.
△ Less
Submitted 17 July, 2024;
originally announced July 2024.
-
Event-Based Simulation of Stochastic Memristive Devices for Neuromorphic Computing
Authors:
Waleed El-Geresy,
Christos Papavassiliou,
Deniz Gündüz
Abstract:
In this paper, we build a general model of memristors suitable for the simulation of event-based systems, such as hardware spiking neural networks, and more generally, neuromorphic computing systems. We extend an existing general model of memristors - the Generalised Metastable Switch Model - to an event-driven setting, eliminating errors associated discrete time approximation, as well as offering…
▽ More
In this paper, we build a general model of memristors suitable for the simulation of event-based systems, such as hardware spiking neural networks, and more generally, neuromorphic computing systems. We extend an existing general model of memristors - the Generalised Metastable Switch Model - to an event-driven setting, eliminating errors associated discrete time approximation, as well as offering potential improvements in terms of computational efficiency for simulation. We introduce the notion of a volatility state variable, to allow for the modelling of memory-dependent and dynamic switching behaviour, succinctly capturing and unifying a variety of volatile phenomena present in memristive devices, including state relaxation, structural disruption, Joule heating, and drift acceleration phenomena. We supply a drift dataset for titanium dioxide memristors and introduce a linear conductance model to simulate the drift characteristics, motivated by a proposed physical model of filament growth. We then demonstrate an approach for fitting the parameters of the event-based model to the drift model.
△ Less
Submitted 23 July, 2024; v1 submitted 14 June, 2024;
originally announced July 2024.
-
Capacity-Maximizing Input Symbol Selection for Discrete Memoryless Channels
Authors:
Maximilian Egger,
Rawad Bitar,
Antonia Wachter-Zeh,
Deniz Gündüz,
Nir Weinberger
Abstract:
Motivated by communication systems with constrained complexity, we consider the problem of input symbol selection for discrete memoryless channels (DMCs). Given a DMC, the goal is to find a subset of its input alphabet, so that the optimal input distribution that is only supported on these symbols maximizes the capacity among all other subsets of the same size (or smaller). We observe that the res…
▽ More
Motivated by communication systems with constrained complexity, we consider the problem of input symbol selection for discrete memoryless channels (DMCs). Given a DMC, the goal is to find a subset of its input alphabet, so that the optimal input distribution that is only supported on these symbols maximizes the capacity among all other subsets of the same size (or smaller). We observe that the resulting optimization problem is non-concave and non-submodular, and so generic methods for such cases do not have theoretical guarantees. We derive an analytical upper bound on the capacity loss when selecting a subset of input symbols based only on the properties of the transition matrix of the channel. We propose a selection algorithm that is based on input-symbols clustering, and an appropriate choice of representatives for each cluster, which uses the theoretical bound as a surrogate objective function. We provide numerical experiments to support the findings.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
Blind Super-Resolution via Meta-learning and Markov Chain Monte Carlo Simulation
Authors:
Jingyuan Xia,
Zhixiong Yang,
Shengxi Li,
Shuanghui Zhang,
Yaowen Fu,
Deniz Gündüz,
Xiang Li
Abstract:
Learning-based approaches have witnessed great successes in blind single image super-resolution (SISR) tasks, however, handcrafted kernel priors and learning based kernel priors are typically required. In this paper, we propose a Meta-learning and Markov Chain Monte Carlo (MCMC) based SISR approach to learn kernel priors from organized randomness. In concrete, a lightweight network is adopted as k…
▽ More
Learning-based approaches have witnessed great successes in blind single image super-resolution (SISR) tasks, however, handcrafted kernel priors and learning based kernel priors are typically required. In this paper, we propose a Meta-learning and Markov Chain Monte Carlo (MCMC) based SISR approach to learn kernel priors from organized randomness. In concrete, a lightweight network is adopted as kernel generator, and is optimized via learning from the MCMC simulation on random Gaussian distributions. This procedure provides an approximation for the rational blur kernel, and introduces a network-level Langevin dynamics into SISR optimization processes, which contributes to preventing bad local optimal solutions for kernel estimation. Meanwhile, a meta-learning-based alternating optimization procedure is proposed to optimize the kernel generator and image restorer, respectively. In contrast to the conventional alternating minimization strategy, a meta-learning-based framework is applied to learn an adaptive optimization strategy, which is less-greedy and results in better convergence performance. These two procedures are iteratively processed in a plug-and-play fashion, for the first time, realizing a learning-based but plug-and-play blind SISR solution in unsupervised inference. Extensive simulations demonstrate the superior performance and generalization ability of the proposed approach when comparing with state-of-the-arts on synthesis and real-world datasets. The code is available at https://github.com/XYLGroup/MLMC.
△ Less
Submitted 13 June, 2024;
originally announced June 2024.
-
Massive Digital Over-the-Air Computation for Communication-Efficient Federated Edge Learning
Authors:
Li Qiao,
Zhen Gao,
Mahdi Boloursaz Mashhadi,
Deniz Gündüz
Abstract:
Over-the-air computation (AirComp) is a promising technology converging communication and computation over wireless networks, which can be particularly effective in model training, inference, and more emerging edge intelligence applications. AirComp relies on uncoded transmission of individual signals, which are added naturally over the multiple access channel thanks to the superposition property…
▽ More
Over-the-air computation (AirComp) is a promising technology converging communication and computation over wireless networks, which can be particularly effective in model training, inference, and more emerging edge intelligence applications. AirComp relies on uncoded transmission of individual signals, which are added naturally over the multiple access channel thanks to the superposition property of the wireless medium. Despite significantly improved communication efficiency, how to accommodate AirComp in the existing and future digital communication networks, that are based on discrete modulation schemes, remains a challenge. This paper proposes a massive digital AirComp (MD-AirComp) scheme, that leverages an unsourced massive access protocol, to enhance compatibility with both current and next-generation wireless networks. MD-AirComp utilizes vector quantization to reduce the uplink communication overhead, and employs shared quantization and modulation codebooks. At the receiver, we propose a near-optimal approximate message passing-based algorithm to compute the model aggregation results from the superposed sequences, which relies on estimating the number of devices transmitting each code sequence, rather than trying to decode the messages of individual transmitters. We apply MD-AirComp to the federated edge learning (FEEL), and show that it significantly accelerates FEEL convergence compared to state-of-the-art while using the same amount of communication resources. To support further research and ensure reproducibility, we have made our code available at https://github.com/liqiao19/MD-AirComp.
△ Less
Submitted 29 August, 2024; v1 submitted 24 May, 2024;
originally announced May 2024.
-
Distributed Indirect Source Coding with Decoder Side Information
Authors:
Jiancheng Tang,
Qianqian Yang,
Deniz Gündüz
Abstract:
This paper studies a variant of the rate-distortion problem motivated by task-oriented semantic communication and distributed learning problems, where $M$ correlated sources are independently encoded for a central decoder. The decoder has access to a correlated side information in addition to the messages received from the encoders, and aims to recover a latent random variable correlated with the…
▽ More
This paper studies a variant of the rate-distortion problem motivated by task-oriented semantic communication and distributed learning problems, where $M$ correlated sources are independently encoded for a central decoder. The decoder has access to a correlated side information in addition to the messages received from the encoders, and aims to recover a latent random variable correlated with the sources observed by the encoders within a given distortion constraint rather than recovering the sources themselves. We provide bounds on the rate-distortion region for this scenario in general, and characterize the rate-distortion function exactly when the sources are conditionally independent given the side information.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
Pragmatic Communication for Remote Control of Finite-State Markov Processes
Authors:
Pietro Talli,
Edoardo David Santi,
Federico Chiariotti,
Touraj Soleymani,
Federico Mason,
Andrea Zanella,
Deniz Gündüz
Abstract:
Pragmatic or goal-oriented communication can optimize communication decisions beyond the reliable transmission of data, instead aiming at directly affecting application performance with the minimum channel utilization. In this paper, we develop a general theoretical framework for the remote control of finite-state Markov processes, using pragmatic communication over a costly zero-delay communicati…
▽ More
Pragmatic or goal-oriented communication can optimize communication decisions beyond the reliable transmission of data, instead aiming at directly affecting application performance with the minimum channel utilization. In this paper, we develop a general theoretical framework for the remote control of finite-state Markov processes, using pragmatic communication over a costly zero-delay communication channel. To that end, we model a cyber-physical system composed of an encoder, which observes and transmits the states of a process in real-time, and a decoder, which receives that information and controls the behavior of the process. The encoder and the decoder should cooperatively optimize the trade-off between the control performance (i.e., reward) and the communication cost (i.e., channel use). This scenario underscores a pragmatic (i.e., goal-oriented) communication problem, where the purpose is to convey only the data that is most valuable for the underlying task, taking into account the state of the decoder (hence, the pragmatic aspect). We investigate two different decision-making architectures: in pull-based remote control, the decoder is the only decision-maker, while in push-based remote control, the encoder and the decoder constitute two independent decision-makers, leading to a multi-agent scenario. We propose three algorithms to optimize our system (i.e., design the encoder and the decoder policies), discuss the optimality guarantees ofs the algorithms, and shed light on their computational complexity and fundamental limits.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
Networked Control with Hybrid Automatic Repeat Request Protocols
Authors:
Touraj Soleymani,
John S. Baras,
Deniz Gündüz
Abstract:
We study feedback control of a dynamical process over a lossy channel equipped with a hybrid automatic repeat request protocol that connects a sensor to an actuator. The dynamical process is modeled by a Gauss-Markov process, and the lossy channel by a packet-erasure channel with ideal feedback. We suppose that data is communicated in the format of packets with negligible quantization error. In su…
▽ More
We study feedback control of a dynamical process over a lossy channel equipped with a hybrid automatic repeat request protocol that connects a sensor to an actuator. The dynamical process is modeled by a Gauss-Markov process, and the lossy channel by a packet-erasure channel with ideal feedback. We suppose that data is communicated in the format of packets with negligible quantization error. In such a networked control system, whenever a packet loss occurs, there exists a tradeoff between transmitting new sensory information with a lower success probability and retransmitting previously failed sensory information with a higher success probability. In essence, an inherent tradeoff between freshness and reliability. To address this tradeoff, we consider a linear-quadratic-regulator performance index, which penalizes state deviations and control efforts over a finite horizon, and jointly design optimal policies for an encoder and a decoder, which are collocated with the sensor and the actuator, respectively. Our emphasis here lies specifically on designing switching and control policies, rather than error-correcting codes. We derive the structural properties of the optimal encoding and decoding policies. We show that the former is a threshold switching policy and the latter is a certainty-equivalent control policy. In addition, we specify the iterative equations that the encoder and the decoder need to solve in order to implement the optimal policies.
△ Less
Submitted 12 May, 2024;
originally announced May 2024.
-
Aggressive or Imperceptible, or Both: Network Pruning Assisted Hybrid Byzantines in Federated Learning
Authors:
Emre Ozfatura,
Kerem Ozfatura,
Alptekin Kupcu,
Deniz Gunduz
Abstract:
Federated learning (FL) has been introduced to enable a large number of clients, possibly mobile devices, to collaborate on generating a generalized machine learning model thanks to utilizing a larger number of local samples without sharing to offer certain privacy to collaborating clients. However, due to the participation of a large number of clients, it is often difficult to profile and verify…
▽ More
Federated learning (FL) has been introduced to enable a large number of clients, possibly mobile devices, to collaborate on generating a generalized machine learning model thanks to utilizing a larger number of local samples without sharing to offer certain privacy to collaborating clients. However, due to the participation of a large number of clients, it is often difficult to profile and verify each client, which leads to a security threat that malicious participants may hamper the accuracy of the trained model by conveying poisoned models during the training. Hence, the aggregation framework at the parameter server also needs to minimize the detrimental effects of these malicious clients. A plethora of attack and defence strategies have been analyzed in the literature. However, often the Byzantine problem is analyzed solely from the outlier detection perspective, being oblivious to the topology of neural networks (NNs).
In the scope of this work, we argue that by extracting certain side information specific to the NN topology, one can design stronger attacks. Hence, inspired by the sparse neural networks, we introduce a hybrid sparse Byzantine attack that is composed of two parts: one exhibiting a sparse nature and attacking only certain NN locations with higher sensitivity, and the other being more silent but accumulating over time, where each ideally targets a different type of defence mechanism, and together they form a strong but imperceptible attack. Finally, we show through extensive simulations that the proposed hybrid Byzantine attack is effective against 8 different defence methods.
△ Less
Submitted 9 April, 2024;
originally announced April 2024.
-
The Rate-Distortion-Perception Trade-off: The Role of Private Randomness
Authors:
Yassine Hamdi,
Aaron B. Wagner,
Deniz Gündüz
Abstract:
In image compression, with recent advances in generative modeling, the existence of a trade-off between the rate and the perceptual quality (realism) has been brought to light, where the realism is measured by the closeness of the output distribution to the source. It has been shown that randomized codes can be strictly better under a number of formulations. In particular, the role of common rando…
▽ More
In image compression, with recent advances in generative modeling, the existence of a trade-off between the rate and the perceptual quality (realism) has been brought to light, where the realism is measured by the closeness of the output distribution to the source. It has been shown that randomized codes can be strictly better under a number of formulations. In particular, the role of common randomness has been well studied. We elucidate the role of private randomness in the compression of a memoryless source $X^n=(X_1,...,X_n)$ under two kinds of realism constraints. The near-perfect realism constraint requires the joint distribution of output symbols $(Y_1,...,Y_n)$ to be arbitrarily close the distribution of the source in total variation distance (TVD). The per-symbol near-perfect realism constraint requires that the TVD between the distribution of output symbol $Y_t$ and the source distribution be arbitrarily small, uniformly in the index $t.$ We characterize the corresponding asymptotic rate-distortion trade-off and show that encoder private randomness is not useful if the compression rate is lower than the entropy of the source, however limited the resources in terms of common randomness and decoder private randomness may be.
△ Less
Submitted 1 April, 2024;
originally announced April 2024.
-
MIMO Channel as a Neural Function: Implicit Neural Representations for Extreme CSI Compression in Massive MIMO Systems
Authors:
Haotian Wu,
Maojun Zhang,
Yulin Shao,
Krystian Mikolajczyk,
Deniz Gündüz
Abstract:
Acquiring and utilizing accurate channel state information (CSI) can significantly improve transmission performance, thereby holding a crucial role in realizing the potential advantages of massive multiple-input multiple-output (MIMO) technology. Current prevailing CSI feedback approaches improve precision by employing advanced deep-learning methods to learn representative CSI features for a subse…
▽ More
Acquiring and utilizing accurate channel state information (CSI) can significantly improve transmission performance, thereby holding a crucial role in realizing the potential advantages of massive multiple-input multiple-output (MIMO) technology. Current prevailing CSI feedback approaches improve precision by employing advanced deep-learning methods to learn representative CSI features for a subsequent compression process. Diverging from previous works, we treat the CSI compression problem in the context of implicit neural representations. Specifically, each CSI matrix is viewed as a neural function that maps the CSI coordinates (antenna number and subchannel) to the corresponding channel gains. Instead of transmitting the parameters of the implicit neural functions directly, we transmit modulations based on the CSI matrix derived through a meta-learning algorithm. Modulations are then applied to a shared base network to generate the elements of the CSI matrix. Modulations corresponding to the CSI matrix are quantized and entropy-coded to further reduce the communication bandwidth, thus achieving extreme CSI compression ratios. Numerical results show that our proposed approach achieves state-of-the-art performance and showcases flexibility in feedback strategies.
△ Less
Submitted 20 March, 2024;
originally announced March 2024.
-
Process-and-Forward: Deep Joint Source-Channel Coding Over Cooperative Relay Networks
Authors:
Chenghong Bian,
Yulin Shao,
Haotian Wu,
Emre Ozfatura,
Deniz Gunduz
Abstract:
This paper introduces an innovative deep joint source-channel coding (DeepJSCC) approach to image transmission over a cooperative relay channel. The relay either amplifies and forwards a scaled version of its received signal, referred to as DeepJSCC-AF, or leverages neural networks to extract relevant features about the source signal before forwarding it to the destination, which we call DeepJSCC-…
▽ More
This paper introduces an innovative deep joint source-channel coding (DeepJSCC) approach to image transmission over a cooperative relay channel. The relay either amplifies and forwards a scaled version of its received signal, referred to as DeepJSCC-AF, or leverages neural networks to extract relevant features about the source signal before forwarding it to the destination, which we call DeepJSCC-PF (Process-and-Forward). In the full-duplex scheme, inspired by the block Markov coding (BMC) concept, we introduce a novel block transmission strategy built upon novel vision transformer architecture. In the proposed scheme, the source transmits information in blocks, and the relay updates its knowledge about the input signal after each block and generates its own signal to be conveyed to the destination. To enhance practicality, we introduce an adaptive transmission model, which allows a single trained DeepJSCC model to adapt seamlessly to various channel qualities, making it a versatile solution. Simulation results demonstrate the superior performance of our proposed DeepJSCC compared to the state-of-the-art BPG image compression algorithm, even when operating at the maximum achievable rate of conventional decode-and-forward and compress-and-forward protocols, for both half-duplex and full-duplex relay scenarios.
△ Less
Submitted 15 March, 2024;
originally announced March 2024.
-
Optimal Denial-of-Service Attacks Against Status Updating
Authors:
Saad Kriouile,
Mohamad Assaad,
Deniz Gündüz,
Touraj Soleymani
Abstract:
In this paper, we investigate denial-of-service attacks against status updating. The target system is modeled by a Markov chain and an unreliable wireless channel, and the performance of status updating in the target system is measured based on two metrics: age of information and age of incorrect information. Our objective is to devise optimal attack policies that strike a balance between the dete…
▽ More
In this paper, we investigate denial-of-service attacks against status updating. The target system is modeled by a Markov chain and an unreliable wireless channel, and the performance of status updating in the target system is measured based on two metrics: age of information and age of incorrect information. Our objective is to devise optimal attack policies that strike a balance between the deterioration of the system's performance and the adversary's energy. We model the optimal problem as a Markov decision process and prove rigorously that the optimal jamming policy is a threshold-based policy under both metrics. In addition, we provide a low-complexity algorithm to obtain the optimal threshold value of the jamming policy. Our numerical results show that the networked system with the age-of-incorrect-information metric is less sensitive to jamming attacks than with the age-of-information metric. Index Terms-age of incorrect information, age of information, cyber-physical systems, status updating, remote monitoring.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
Extreme Video Compression with Pre-trained Diffusion Models
Authors:
Bohan Li,
Yiming Liu,
Xueyan Niu,
Bo Bai,
Lei Deng,
Deniz Gündüz
Abstract:
Diffusion models have achieved remarkable success in generating high quality image and video data. More recently, they have also been used for image compression with high perceptual quality. In this paper, we present a novel approach to extreme video compression leveraging the predictive power of diffusion-based generative models at the decoder. The conditional diffusion model takes several neural…
▽ More
Diffusion models have achieved remarkable success in generating high quality image and video data. More recently, they have also been used for image compression with high perceptual quality. In this paper, we present a novel approach to extreme video compression leveraging the predictive power of diffusion-based generative models at the decoder. The conditional diffusion model takes several neural compressed frames and generates subsequent frames. When the reconstruction quality drops below the desired level, new frames are encoded to restart prediction. The entire video is sequentially encoded to achieve a visually pleasing reconstruction, considering perceptual quality metrics such as the learned perceptual image patch similarity (LPIPS) and the Frechet video distance (FVD), at bit rates as low as 0.02 bits per pixel (bpp). Experimental results demonstrate the effectiveness of the proposed scheme compared to standard codecs such as H.264 and H.265 in the low bpp regime. The results showcase the potential of exploiting the temporal relations in video data using generative models. Code is available at: https://github.com/ElesionKyrie/Extreme-Video-Compression-With-Prediction-Using-Pre-trainded-Diffusion-Models-
△ Less
Submitted 13 February, 2024;
originally announced February 2024.
-
Full-Duplex Millimeter Wave MIMO Channel Estimation: A Neural Network Approach
Authors:
Mehdi Sattari,
Hao Guo,
Deniz Gündüz,
Ashkan Panahi,
Tommy Svensson
Abstract:
Millimeter wave (mmWave) multiple-input-multi-output (MIMO) is now a reality with great potential for further improvement. We study full-duplex transmissions as an effective way to improve mmWave MIMO systems. Compared to half-duplex systems, full-duplex transmissions may offer higher data rates and lower latency. However, full-duplex transmission is hindered by self-interference (SI) at the recei…
▽ More
Millimeter wave (mmWave) multiple-input-multi-output (MIMO) is now a reality with great potential for further improvement. We study full-duplex transmissions as an effective way to improve mmWave MIMO systems. Compared to half-duplex systems, full-duplex transmissions may offer higher data rates and lower latency. However, full-duplex transmission is hindered by self-interference (SI) at the receive antennas, and SI channel estimation becomes a crucial step to make the full-duplex systems feasible. In this paper, we address the problem of channel estimation in full-duplex mmWave MIMO systems using neural networks (NNs). Our approach involves sharing pilot resources between user equipments (UEs) and transmit antennas at the base station (BS), aiming to reduce the pilot overhead in full-duplex systems and to achieve a comparable level to that of a half-duplex system. Additionally, in the case of separate antenna configurations in a full-duplex BS, providing channel estimates of transmit antenna (TX) arrays to the downlink UEs poses another challenge, as the TX arrays are not capable of receiving pilot signals. To address this, we employ an NN to map the channel from the downlink UEs to the receive antenna (RX) arrays to the channel from the TX arrays to the downlink UEs. We further elaborate on how NNs perform the estimation with different architectures, (e.g., different numbers of hidden layers), the introduction of non-linear distortion (e.g., with a 1-bit analog-to-digital converter (ADC)), and different channel conditions (e.g., low-correlated and high-correlated channels). Our work provides novel insights into NN-based channel estimators.
△ Less
Submitted 18 June, 2024; v1 submitted 6 February, 2024;
originally announced February 2024.
-
Friendly Attacks to Improve Channel Coding Reliability
Authors:
Anastasiia Kurmukova,
Deniz Gunduz
Abstract:
This paper introduces a novel approach called "friendly attack" aimed at enhancing the performance of error correction channel codes. Inspired by the concept of adversarial attacks, our method leverages the idea of introducing slight perturbations to the neural network input, resulting in a substantial impact on the network's performance. By introducing small perturbations to fixed-point modulated…
▽ More
This paper introduces a novel approach called "friendly attack" aimed at enhancing the performance of error correction channel codes. Inspired by the concept of adversarial attacks, our method leverages the idea of introducing slight perturbations to the neural network input, resulting in a substantial impact on the network's performance. By introducing small perturbations to fixed-point modulated codewords before transmission, we effectively improve the decoder's performance without violating the input power constraint. The perturbation design is accomplished by a modified iterative fast gradient method. This study investigates various decoder architectures suitable for computing gradients to obtain the desired perturbations. Specifically, we consider belief propagation (BP) for LDPC codes; the error correcting code transformer, BP and neural BP (NBP) for polar codes, and neural BCJR for convolutional codes. We demonstrate that the proposed friendly attack method can improve the reliability across different channels, modulations, codes, and decoders. This method allows us to increase the reliability of communication with a legacy receiver by simply modifying the transmitted codeword appropriately.
△ Less
Submitted 25 January, 2024;
originally announced January 2024.
-
Maximal-Capacity Discrete Memoryless Channel Identification
Authors:
Maximilian Egger,
Rawad Bitar,
Antonia Wachter-Zeh,
Deniz Gündüz,
Nir Weinberger
Abstract:
The problem of identifying the channel with the highest capacity among several discrete memoryless channels (DMCs) is considered. The problem is cast as a pure-exploration multi-armed bandit problem, which follows the practical use of training sequences to sense the communication channel statistics. A capacity estimator is proposed and tight confidence bounds on the estimator error are derived. Ba…
▽ More
The problem of identifying the channel with the highest capacity among several discrete memoryless channels (DMCs) is considered. The problem is cast as a pure-exploration multi-armed bandit problem, which follows the practical use of training sequences to sense the communication channel statistics. A capacity estimator is proposed and tight confidence bounds on the estimator error are derived. Based on this capacity estimator, a gap-elimination algorithm termed BestChanID is proposed, which is oblivious to the capacity-achieving input distribution and is guaranteed to output the DMC with the largest capacity, with a desired confidence. Furthermore, two additional algorithms NaiveChanSel and MedianChanEl, that output with certain confidence a DMC with capacity close to the maximal, are introduced. Each of those algorithms is beneficial in a different regime and can be used as a subroutine in BestChanID. The sample complexity of all algorithms is analyzed as a function of the desired confidence parameter, the number of channels, and the channels' input and output alphabet sizes. The cost of best channel identification is shown to scale quadratically with the alphabet size, and a fundamental lower bound for the required number of channel senses to identify the best channel with a certain confidence is derived.
△ Less
Submitted 18 January, 2024;
originally announced January 2024.
-
Mobility Accelerates Learning: Convergence Analysis on Hierarchical Federated Learning in Vehicular Networks
Authors:
Tan Chen,
Jintao Yan,
Yuxuan Sun,
Sheng Zhou,
Deniz Gündüz,
Zhisheng Niu
Abstract:
Hierarchical federated learning (HFL) enables distributed training of models across multiple devices with the help of several edge servers and a cloud edge server in a privacy-preserving manner. In this paper, we consider HFL with highly mobile devices, mainly targeting at vehicular networks. Through convergence analysis, we show that mobility influences the convergence speed by both fusing the ed…
▽ More
Hierarchical federated learning (HFL) enables distributed training of models across multiple devices with the help of several edge servers and a cloud edge server in a privacy-preserving manner. In this paper, we consider HFL with highly mobile devices, mainly targeting at vehicular networks. Through convergence analysis, we show that mobility influences the convergence speed by both fusing the edge data and shuffling the edge models. While mobility is usually considered as a challenge from the perspective of communication, we prove that it increases the convergence speed of HFL with edge-level heterogeneous data, since more diverse data can be incorporated. Furthermore, we demonstrate that a higher speed leads to faster convergence, since it accelerates the fusion of data. Simulation results show that mobility increases the model accuracy of HFL by up to 15.1% when training a convolutional neural network on the CIFAR-10 dataset.
△ Less
Submitted 17 January, 2024;
originally announced January 2024.
-
Point Cloud in the Air
Authors:
Yulin Shao,
Chenghong Bian,
Li Yang,
Qianqian Yang,
Zhaoyang Zhang,
Deniz Gunduz
Abstract:
Acquisition and processing of point clouds (PCs) is a crucial enabler for many emerging applications reliant on 3D spatial data, such as robot navigation, autonomous vehicles, and augmented reality. In most scenarios, PCs acquired by remote sensors must be transmitted to an edge server for fusion, segmentation, or inference. Wireless transmission of PCs not only puts on increased burden on the alr…
▽ More
Acquisition and processing of point clouds (PCs) is a crucial enabler for many emerging applications reliant on 3D spatial data, such as robot navigation, autonomous vehicles, and augmented reality. In most scenarios, PCs acquired by remote sensors must be transmitted to an edge server for fusion, segmentation, or inference. Wireless transmission of PCs not only puts on increased burden on the already congested wireless spectrum, but also confronts a unique set of challenges arising from the irregular and unstructured nature of PCs. In this paper, we meticulously delineate these challenges and offer a comprehensive examination of existing solutions while candidly acknowledging their inherent limitations. In response to these intricacies, we proffer four pragmatic solution frameworks, spanning advanced techniques, hybrid schemes, and distributed data aggregation approaches. In doing so, our goal is to chart a path toward efficient, reliable, and low-latency wireless PC transmission.
△ Less
Submitted 31 December, 2023;
originally announced January 2024.
-
Adaptive Early Exiting for Collaborative Inference over Noisy Wireless Channels
Authors:
Mikolaj Jankowski,
Deniz Gunduz,
Krystian Mikolajczyk
Abstract:
Collaborative inference systems are one of the emerging solutions for deploying deep neural networks (DNNs) at the wireless network edge. Their main idea is to divide a DNN into two parts, where the first is shallow enough to be reliably executed at edge devices of limited computational power, while the second part is executed at an edge server with higher computational capabilities. The main adva…
▽ More
Collaborative inference systems are one of the emerging solutions for deploying deep neural networks (DNNs) at the wireless network edge. Their main idea is to divide a DNN into two parts, where the first is shallow enough to be reliably executed at edge devices of limited computational power, while the second part is executed at an edge server with higher computational capabilities. The main advantage of such systems is that the input of the DNN gets compressed as the subsequent layers of the shallow part extract only the information necessary for the task. As a result, significant communication savings can be achieved compared to transmitting raw input samples. In this work, we study early exiting in the context of collaborative inference, which allows obtaining inference results at the edge device for certain samples, without the need to transmit the partially processed data to the edge server at all, leading to further communication savings. The central part of our system is the transmission-decision (TD) mechanism, which, given the information from the early exit, and the wireless channel conditions, decides whether to keep the early exit prediction or transmit the data to the edge server for further processing. In this paper, we evaluate various TD mechanisms and show experimentally, that for an image classification task over the wireless edge, proper utilization of early exits can provide both performance gains and significant communication savings.
△ Less
Submitted 29 November, 2023;
originally announced November 2023.
-
Multi-Agent Reinforcement Learning for Power Control in Wireless Networks via Adaptive Graphs
Authors:
Lorenzo Mario Amorosa,
Marco Skocaj,
Roberto Verdone,
Deniz Gündüz
Abstract:
The ever-increasing demand for high-quality and heterogeneous wireless communication services has driven extensive research on dynamic optimization strategies in wireless networks. Among several possible approaches, multi-agent deep reinforcement learning (MADRL) has emerged as a promising method to address a wide range of complex optimization problems like power control. However, the seamless app…
▽ More
The ever-increasing demand for high-quality and heterogeneous wireless communication services has driven extensive research on dynamic optimization strategies in wireless networks. Among several possible approaches, multi-agent deep reinforcement learning (MADRL) has emerged as a promising method to address a wide range of complex optimization problems like power control. However, the seamless application of MADRL to a variety of network optimization problems faces several challenges related to convergence. In this paper, we present the use of graphs as communication-inducing structures among distributed agents as an effective means to mitigate these challenges. Specifically, we harness graph neural networks (GNNs) as neural architectures for policy parameterization to introduce a relational inductive bias in the collective decision-making process. Most importantly, we focus on modeling the dynamic interactions among sets of neighboring agents through the introduction of innovative methods for defining a graph-induced framework for integrated communication and learning. Finally, the superior generalization capabilities of the proposed methodology to larger networks and to networks with different user categories is verified through simulations.
△ Less
Submitted 27 November, 2023;
originally announced November 2023.
-
Private Variable-Length Coding with Zero Leakage
Authors:
Amirreza Zamani,
Tobias J. Oechtering,
Deniz Gündüz,
Mikael Skoglund
Abstract:
A private compression design problem is studied, where an encoder observes useful data $Y$, wishes to compress it using variable length code and communicates it through an unsecured channel. Since $Y$ is correlated with private attribute $X$, the encoder uses a private compression mechanism to design encoded message $\cal C$ and sends it over the channel. An adversary is assumed to have access to…
▽ More
A private compression design problem is studied, where an encoder observes useful data $Y$, wishes to compress it using variable length code and communicates it through an unsecured channel. Since $Y$ is correlated with private attribute $X$, the encoder uses a private compression mechanism to design encoded message $\cal C$ and sends it over the channel. An adversary is assumed to have access to the output of the encoder, i.e., $\cal C$, and tries to estimate $X$. Furthermore, it is assumed that both encoder and decoder have access to a shared secret key $W$. The design goal is to encode message $\cal C$ with minimum possible average length that satisfies a perfect privacy constraint. To do so we first consider two different privacy mechanism design problems and find upper bounds on the entropy of the optimizers by solving a linear program. We use the obtained optimizers to design $\cal C$. In two cases we strengthen the existing bounds: 1. $|\mathcal{X}|\geq |\mathcal{Y}|$; 2. The realization of $(X,Y)$ follows a specific joint distribution. In particular, considering the second case we use two-part construction coding to achieve the upper bounds. Furthermore, in a numerical example we study the obtained bounds and show that they can improve the existing results.
△ Less
Submitted 29 October, 2023;
originally announced October 2023.
-
Distributed Deep Joint Source-Channel Coding with Decoder-Only Side Information
Authors:
Selim F. Yilmaz,
Ezgi Ozyilkan,
Deniz Gunduz,
Elza Erkip
Abstract:
We consider low-latency image transmission over a noisy wireless channel when correlated side information is present only at the receiver side (the Wyner-Ziv scenario). In particular, we are interested in developing practical schemes using a data-driven joint source-channel coding (JSCC) approach, which has been previously shown to outperform conventional separation-based approaches in the practic…
▽ More
We consider low-latency image transmission over a noisy wireless channel when correlated side information is present only at the receiver side (the Wyner-Ziv scenario). In particular, we are interested in developing practical schemes using a data-driven joint source-channel coding (JSCC) approach, which has been previously shown to outperform conventional separation-based approaches in the practical finite blocklength regimes, and to provide graceful degradation with channel quality. We propose a novel neural network architecture that incorporates the decoder-only side information at multiple stages at the receiver side. Our results demonstrate that the proposed method succeeds in integrating the side information, yielding improved performance at all channel conditions in terms of the various quality measures considered here, especially at low channel signal-to-noise ratios (SNRs) and small bandwidth ratios (BRs). We have made the source code of the proposed method public to enable further research, and the reproducibility of the results.
△ Less
Submitted 27 February, 2024; v1 submitted 6 October, 2023;
originally announced October 2023.
-
Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework
Authors:
Jingheng Zheng,
Wanli Ni,
Hui Tian,
Deniz Gunduz,
Tony Q. S. Quek,
Zhu Han
Abstract:
Under the organization of the base station (BS), wireless federated learning (FL) enables collaborative model training among multiple devices. However, the BS is merely responsible for aggregating local updates during the training process, which incurs a waste of the computational resource at the BS. To tackle this issue, we propose a semi-federated learning (SemiFL) paradigm to leverage the compu…
▽ More
Under the organization of the base station (BS), wireless federated learning (FL) enables collaborative model training among multiple devices. However, the BS is merely responsible for aggregating local updates during the training process, which incurs a waste of the computational resource at the BS. To tackle this issue, we propose a semi-federated learning (SemiFL) paradigm to leverage the computing capabilities of both the BS and devices for a hybrid implementation of centralized learning (CL) and FL. Specifically, each device sends both local gradients and data samples to the BS for training a shared global model. To improve communication efficiency over the same time-frequency resources, we integrate over-the-air computation for aggregation and non-orthogonal multiple access for transmission by designing a novel transceiver structure. To gain deep insights, we conduct convergence analysis by deriving a closed-form optimality gap for SemiFL and extend the result to two extra cases. In the first case, the BS uses all accumulated data samples to calculate the CL gradient, while a decreasing learning rate is adopted in the second case. Our analytical results capture the destructive effect of wireless communication and show that both FL and CL are special cases of SemiFL. Then, we formulate a non-convex problem to reduce the optimality gap by jointly optimizing the transmit power and receive beamformers. Accordingly, we propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers. Extensive simulation results on two real-world datasets corroborate our theoretical analysis, and show that the proposed SemiFL outperforms conventional FL and achieves 3.2% accuracy gain on the MNIST dataset compared to state-of-the-art benchmarks.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
CommIN: Semantic Image Communications as an Inverse Problem with INN-Guided Diffusion Models
Authors:
Jiakang Chen,
Di You,
Deniz Gündüz,
Pier Luigi Dragotti
Abstract:
Joint source-channel coding schemes based on deep neural networks (DeepJSCC) have recently achieved remarkable performance for wireless image transmission. However, these methods usually focus only on the distortion of the reconstructed signal at the receiver side with respect to the source at the transmitter side, rather than the perceptual quality of the reconstruction which carries more semanti…
▽ More
Joint source-channel coding schemes based on deep neural networks (DeepJSCC) have recently achieved remarkable performance for wireless image transmission. However, these methods usually focus only on the distortion of the reconstructed signal at the receiver side with respect to the source at the transmitter side, rather than the perceptual quality of the reconstruction which carries more semantic information. As a result, severe perceptual distortion can be introduced under extreme conditions such as low bandwidth and low signal-to-noise ratio. In this work, we propose CommIN, which views the recovery of high-quality source images from degraded reconstructions as an inverse problem. To address this, CommIN combines Invertible Neural Networks (INN) with diffusion models, aiming for superior perceptual quality. Through experiments, we show that our CommIN significantly improves the perceptual quality compared to DeepJSCC under extreme conditions and outperforms other inverse problem approaches used in DeepJSCC.
△ Less
Submitted 2 October, 2023;
originally announced October 2023.
-
High Perceptual Quality Wireless Image Delivery with Denoising Diffusion Models
Authors:
Selim F. Yilmaz,
Xueyan Niu,
Bo Bai,
Wei Han,
Lei Deng,
Deniz Gunduz
Abstract:
We consider the image transmission problem over a noisy wireless channel via deep learning-based joint source-channel coding (DeepJSCC) along with a denoising diffusion probabilistic model (DDPM) at the receiver. Specifically, we are interested in the perception-distortion trade-off in the practical finite block length regime, in which separate source and channel coding can be highly suboptimal. W…
▽ More
We consider the image transmission problem over a noisy wireless channel via deep learning-based joint source-channel coding (DeepJSCC) along with a denoising diffusion probabilistic model (DDPM) at the receiver. Specifically, we are interested in the perception-distortion trade-off in the practical finite block length regime, in which separate source and channel coding can be highly suboptimal. We introduce a novel scheme that utilizes the range-null space decomposition of the target image. We transmit the range-space of the image after encoding and employ DDPM to progressively refine its null space contents. Through extensive experiments, we demonstrate significant improvements in distortion and perceptual quality of reconstructed images compared to standard DeepJSCC and the state-of-the-art generative learning-based method. We will publicly share our source code to facilitate further research and reproducibility.
△ Less
Submitted 27 September, 2023;
originally announced September 2023.
-
Private Variable-Length Coding with Sequential Encoder
Authors:
Amirreza Zamani,
Tobias J. Oechtering,
Deniz Gündüz,
Mikael Skoglund
Abstract:
A multi-user private data compression problem is studied. A server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is connected to an encoder. The encoder is connected through an unsecured link to a user. We assume that each file $Y_i$ is arbitrarily correlated with a private attribute $X$, which is assumed to be accessible by the encoder. Moreover, an adversary i…
▽ More
A multi-user private data compression problem is studied. A server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is connected to an encoder. The encoder is connected through an unsecured link to a user. We assume that each file $Y_i$ is arbitrarily correlated with a private attribute $X$, which is assumed to be accessible by the encoder. Moreover, an adversary is assumed to have access to the link. The users and the encoder have access to a shared secret key $W$. We assume that at each time the user asks for a file $Y_{d_i}$, where $(d_1,\ldots,d_K)$ corresponds to the demand vector. The goal is to design the delivered message $\mathcal {C}=(\mathcal {C}_1,\ldots,\mathcal {C}_K)$ after the user send his demands to the encoder such that the average length of $\mathcal{C}$ is minimized, while satisfying: i. The message $\cal C$ does not reveal any information about $X$, i.e., $X$ and $\mathcal{C}$ are independent, which corresponds to the perfect privacy constraint; ii. The user is able to decode its demands, $Y_{d_i}$, by using $\cal C$, and the shared key $W$. Here, the encoder sequentially encode each demand $Y_{d_i}$ at time $i$, using the shared key and previous encoded messages.
We propose a variable-length coding scheme that uses privacy-aware compression techniques. We study proposed upper and lower bounds on the average length of $\mathcal{C}$ in an example. Finally, we study an application considering cache-aided networks.
△ Less
Submitted 16 September, 2023;
originally announced September 2023.
-
Communication Efficient Private Federated Learning Using Dithering
Authors:
Burak Hasircioglu,
Deniz Gunduz
Abstract:
The task of preserving privacy while ensuring efficient communication is a fundamental challenge in federated learning. In this work, we tackle this challenge in the trusted aggregator model, and propose a solution that achieves both objectives simultaneously. We show that employing a quantization scheme based on subtractive dithering at the clients can effectively replicate the normal noise addit…
▽ More
The task of preserving privacy while ensuring efficient communication is a fundamental challenge in federated learning. In this work, we tackle this challenge in the trusted aggregator model, and propose a solution that achieves both objectives simultaneously. We show that employing a quantization scheme based on subtractive dithering at the clients can effectively replicate the normal noise addition process at the aggregator. This implies that we can guarantee the same level of differential privacy against other clients while substantially reducing the amount of communication required, as opposed to transmitting full precision gradients and using central noise addition. We also experimentally demonstrate that the accuracy of our proposed approach matches that of the full precision gradient method.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
Deep Joint Source-Channel Coding for Adaptive Image Transmission over MIMO Channels
Authors:
Haotian Wu,
Yulin Shao,
Chenghong Bian,
Krystian Mikolajczyk,
Deniz Gündüz
Abstract:
This paper introduces a vision transformer (ViT)-based deep joint source and channel coding (DeepJSCC) scheme for wireless image transmission over multiple-input multiple-output (MIMO) channels, denoted as DeepJSCC-MIMO. We consider DeepJSCC-MIMO for adaptive image transmission in both open-loop and closed-loop MIMO systems. The novel DeepJSCC-MIMO architecture surpasses the classical separation-b…
▽ More
This paper introduces a vision transformer (ViT)-based deep joint source and channel coding (DeepJSCC) scheme for wireless image transmission over multiple-input multiple-output (MIMO) channels, denoted as DeepJSCC-MIMO. We consider DeepJSCC-MIMO for adaptive image transmission in both open-loop and closed-loop MIMO systems. The novel DeepJSCC-MIMO architecture surpasses the classical separation-based benchmarks with robustness to channel estimation errors and showcases remarkable flexibility in adapting to diverse channel conditions and antenna numbers without requiring retraining. Specifically, by harnessing the self-attention mechanism of ViT, DeepJSCC-MIMO intelligently learns feature mapping and power allocation strategies tailored to the unique characteristics of the source image and prevailing channel conditions. Extensive numerical experiments validate the significant improvements in transmission quality achieved by DeepJSCC-MIMO for both open-loop and closed-loop MIMO systems across a wide range of scenarios. Moreover, DeepJSCC-MIMO exhibits robustness to varying channel conditions, channel estimation errors, and different antenna numbers, making it an appealing solution for emerging semantic communication systems.
△ Less
Submitted 14 July, 2024; v1 submitted 1 September, 2023;
originally announced September 2023.
-
State Estimation over Broadcast and Multi-Access Channels in an Unreliable Regime
Authors:
Touraj Soleymani,
Deniz Gündüz
Abstract:
This article examines the problem of state estimation over multi-terminal channels in an unreliable regime. More specifically, we consider two canonical settings. In the first setting, measurements of a common stochastic source need to be transmitted to two distinct remote monitors over a packet-erasure broadcast channel. In the second setting, measurements of two distinct stochastic sources need…
▽ More
This article examines the problem of state estimation over multi-terminal channels in an unreliable regime. More specifically, we consider two canonical settings. In the first setting, measurements of a common stochastic source need to be transmitted to two distinct remote monitors over a packet-erasure broadcast channel. In the second setting, measurements of two distinct stochastic sources need to be transmitted to a common remote monitor over a packet-erasure multi-access channel. For these networked systems, we uncover the fundamental performance limits in the sense of a causal tradeoff between the estimation error and the communication cost by identifying optimal encoding and decoding strategies. In the course of our analysis, we introduce two novel semantic metrics that play essential roles in state estimation over broadcast and multi-access channels. The first metric arising in the context of broadcast channels is the dissemination value of information, which quantifies the valuation of provisioning a piece of information to multiple receivers simultaneously. The second metric arising in the context of multi-access channels is the prioritization value of information, which quantifies the valuation of provisioning a piece of information chosen from one out of multiple transmitters. Our findings certify that the optimal encoding and decoding strategies hinge on these semantic metrics.
△ Less
Submitted 30 August, 2023;
originally announced August 2023.
-
A Hybrid Wireless Image Transmission Scheme with Diffusion
Authors:
Xueyan Niu,
Xu Wang,
Deniz Gündüz,
Bo Bai,
Weichao Chen,
Guohua Zhou
Abstract:
We propose a hybrid joint source-channel coding (JSCC) scheme, in which the conventional digital communication scheme is complemented with a generative refinement component to improve the perceptual quality of the reconstruction. The input image is decomposed into two components: the first is a coarse compressed version, and is transmitted following the conventional separation based approach. An a…
▽ More
We propose a hybrid joint source-channel coding (JSCC) scheme, in which the conventional digital communication scheme is complemented with a generative refinement component to improve the perceptual quality of the reconstruction. The input image is decomposed into two components: the first is a coarse compressed version, and is transmitted following the conventional separation based approach. An additional component is obtained through the diffusion process by adding independent Gaussian noise to the input image, and is transmitted using DeepJSCC. The decoder combines the two signals to produce a high quality reconstruction of the source. Experimental results show that the hybrid design provides bandwidth savings and enables graceful performance improvement as the channel quality improves.
△ Less
Submitted 16 August, 2023;
originally announced August 2023.
-
Secure Deep-JSCC Against Multiple Eavesdroppers
Authors:
Seyyed Amirhossein Ameli Kalkhoran,
Mehdi Letafati,
Ecenaz Erdemir,
Babak Hossein Khalaj,
Hamid Behroozi,
Deniz Gündüz
Abstract:
In this paper, a generalization of deep learning-aided joint source channel coding (Deep-JSCC) approach to secure communications is studied. We propose an end-to-end (E2E) learning-based approach for secure communication against multiple eavesdroppers over complex-valued fading channels. Both scenarios of colluding and non-colluding eavesdroppers are studied. For the colluding strategy, eavesdropp…
▽ More
In this paper, a generalization of deep learning-aided joint source channel coding (Deep-JSCC) approach to secure communications is studied. We propose an end-to-end (E2E) learning-based approach for secure communication against multiple eavesdroppers over complex-valued fading channels. Both scenarios of colluding and non-colluding eavesdroppers are studied. For the colluding strategy, eavesdroppers share their logits to collaboratively infer private attributes based on ensemble learning method, while for the non-colluding setup they act alone. The goal is to prevent eavesdroppers from inferring private (sensitive) information about the transmitted images, while delivering the images to a legitimate receiver with minimum distortion. By generalizing the ideas of privacy funnel and wiretap channel coding, the trade-off between the image recovery at the legitimate node and the information leakage to the eavesdroppers is characterized. To solve this secrecy funnel framework, we implement deep neural networks (DNNs) to realize a data-driven secure communication scheme, without relying on a specific data distribution. Simulations over CIFAR-10 dataset verifies the secrecy-utility trade-off. Adversarial accuracy of eavesdroppers are also studied over Rayleigh fading, Nakagami-m, and AWGN channels to verify the generalization of the proposed scheme. Our experiments show that employing the proposed secure neural encoding can decrease the adversarial accuracy by 28%.
△ Less
Submitted 5 August, 2023;
originally announced August 2023.
-
Timely and Massive Communication in 6G: Pragmatics, Learning, and Inference
Authors:
Deniz Gündüz,
Federico Chiariotti,
Kaibin Huang,
Anders E. Kalør,
Szymon Kobus,
Petar Popovski
Abstract:
5G has expanded the traditional focus of wireless systems to embrace two new connectivity types: ultra-reliable low latency and massive communication. The technology context at the dawn of 6G is different from the past one for 5G, primarily due to the growing intelligence at the communicating nodes. This has driven the set of relevant communication problems beyond reliable transmission towards sem…
▽ More
5G has expanded the traditional focus of wireless systems to embrace two new connectivity types: ultra-reliable low latency and massive communication. The technology context at the dawn of 6G is different from the past one for 5G, primarily due to the growing intelligence at the communicating nodes. This has driven the set of relevant communication problems beyond reliable transmission towards semantic and pragmatic communication. This paper puts the evolution of low-latency and massive communication towards 6G in the perspective of these new developments. At first, semantic/pragmatic communication problems are presented by drawing parallels to linguistics. We elaborate upon the relation of semantic communication to the information-theoretic problems of source/channel coding, while generalized real-time communication is put in the context of cyber-physical systems and real-time inference. The evolution of massive access towards massive closed-loop communication is elaborated upon, enabling interactive communication, learning, and cooperation among wireless sensors and actuators.
△ Less
Submitted 26 September, 2023; v1 submitted 30 June, 2023;
originally announced June 2023.
-
Cache-Aided Private Variable-Length Coding with Zero and Non-Zero Leakage
Authors:
Amirreza Zamani,
Tobias J. Oechtering,
Deniz Gündüz,
Mikael Skoglund
Abstract:
A private cache-aided compression problem is studied, where a server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is connected through a shared link to $K$ users, each equipped with a local cache of size $MF$ bits. In the placement phase, the server fills the users$'$ caches without knowing their demands, while the delivery phase takes place after the users sen…
▽ More
A private cache-aided compression problem is studied, where a server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is connected through a shared link to $K$ users, each equipped with a local cache of size $MF$ bits. In the placement phase, the server fills the users$'$ caches without knowing their demands, while the delivery phase takes place after the users send their demands to the server. We assume that each file $Y_i$ is arbitrarily correlated with a private attribute $X$, and an adversary is assumed to have access to the shared link. The users and the server have access to a shared key $W$. The goal is to design the cache contents and the delivered message $\cal C$ such that the average length of $\mathcal{C}$ is minimized, while satisfying: i. The response $\cal C$ does not reveal any information about $X$, i.e., $X$ and $\cal C$ are independent, which corresponds to the perfect privacy constraint; ii. User $i$ is able to decode its demand, $Y_{d_i}$, by using $\cal C$, its local cache $Z_i$, and the shared key $W$. Since the database is correlated with $X$, existing codes for cache-aided delivery do not satisfy the perfect privacy condition. Indeed, we propose a variable-length coding scheme that combines privacy-aware compression with coded caching techniques. In particular, we use two-part code construction and Functional Representation Lemma. Finally, we extend the results to the case, where $X$ and $\mathcal{C}$ can be correlated, i.e., non-zero leakage is allowed.
△ Less
Submitted 22 June, 2023;
originally announced June 2023.
-
Adaptive Compression in Federated Learning via Side Information
Authors:
Berivan Isik,
Francesco Pase,
Deniz Gunduz,
Sanmi Koyejo,
Tsachy Weissman,
Michele Zorzi
Abstract:
The high communication cost of sending model updates from the clients to the server is a significant bottleneck for scalable federated learning (FL). Among existing approaches, state-of-the-art bitrate-accuracy tradeoffs have been achieved using stochastic compression methods -- in which the client $n$ sends a sample from a client-only probability distribution $q_{φ^{(n)}}$, and the server estimat…
▽ More
The high communication cost of sending model updates from the clients to the server is a significant bottleneck for scalable federated learning (FL). Among existing approaches, state-of-the-art bitrate-accuracy tradeoffs have been achieved using stochastic compression methods -- in which the client $n$ sends a sample from a client-only probability distribution $q_{φ^{(n)}}$, and the server estimates the mean of the clients' distributions using these samples. However, such methods do not take full advantage of the FL setup where the server, throughout the training process, has side information in the form of a global distribution $p_θ$ that is close to the clients' distribution $q_{φ^{(n)}}$ in Kullback-Leibler (KL) divergence. In this work, we exploit this closeness between the clients' distributions $q_{φ^{(n)}}$'s and the side information $p_θ$ at the server, and propose a framework that requires approximately $D_{KL}(q_{φ^{(n)}}|| p_θ)$ bits of communication. We show that our method can be integrated into many existing stochastic compression frameworks to attain the same (and often higher) test accuracy with up to $82$ times smaller bitrate than the prior work -- corresponding to 2,650 times overall compression.
△ Less
Submitted 21 April, 2024; v1 submitted 21 June, 2023;
originally announced June 2023.
-
Data-Heterogeneous Hierarchical Federated Learning with Mobility
Authors:
Tan Chen,
Jintao Yan,
Yuxuan Sun,
Sheng Zhou,
Deniz Gunduz,
Zhisheng Niu
Abstract:
Federated learning enables distributed training of machine learning (ML) models across multiple devices in a privacy-preserving manner. Hierarchical federated learning (HFL) is further proposed to meet the requirements of both latency and coverage. In this paper, we consider a data-heterogeneous HFL scenario with mobility, mainly targeting vehicular networks. We derive the convergence upper bound…
▽ More
Federated learning enables distributed training of machine learning (ML) models across multiple devices in a privacy-preserving manner. Hierarchical federated learning (HFL) is further proposed to meet the requirements of both latency and coverage. In this paper, we consider a data-heterogeneous HFL scenario with mobility, mainly targeting vehicular networks. We derive the convergence upper bound of HFL with respect to mobility and data heterogeneity, and analyze how mobility impacts the performance of HFL. While mobility is considered as a challenge from a communication point of view, our goal here is to exploit mobility to improve the learning performance by mitigating data heterogeneity. Simulation results verify the analysis and show that mobility can indeed improve the model accuracy by up to 15.1\% when training a convolutional neural network on the CIFAR-10 dataset using HFL.
△ Less
Submitted 19 June, 2023;
originally announced June 2023.
-
Transformer-aided Wireless Image Transmission with Channel Feedback
Authors:
Haotian Wu,
Yulin Shao,
Emre Ozfatura,
Krystian Mikolajczyk,
Deniz Gündüz
Abstract:
This paper presents a novel wireless image transmission paradigm that can exploit feedback from the receiver, called DeepJSCC-ViT-f. We consider a block feedback channel model, where the transmitter receives noiseless/noisy channel output feedback after each block. The proposed scheme employs a single encoder to facilitate transmission over multiple blocks, refining the receiver's estimation at ea…
▽ More
This paper presents a novel wireless image transmission paradigm that can exploit feedback from the receiver, called DeepJSCC-ViT-f. We consider a block feedback channel model, where the transmitter receives noiseless/noisy channel output feedback after each block. The proposed scheme employs a single encoder to facilitate transmission over multiple blocks, refining the receiver's estimation at each block. Specifically, the unified encoder of DeepJSCC-ViT-f can leverage the semantic information from the source image, and acquire channel state information and the decoder's current belief about the source image from the feedback signal to generate coded symbols at each block. Numerical experiments show that our DeepJSCC-ViT-f scheme achieves state-of-the-art transmission performance with robustness to noise in the feedback link. Additionally, DeepJSCC-ViT-f can adapt to the channel condition directly through feedback without the need for separate channel estimation. We further extend the scope of the DeepJSCC-ViT-f approach to include the broadcast channel, which enables the transmitter to generate broadcast codes in accordance with signal semantics and channel feedback from individual receivers.
△ Less
Submitted 14 February, 2024; v1 submitted 15 June, 2023;
originally announced June 2023.
-
Wireless Point Cloud Transmission
Authors:
Chenghong Bian,
Yulin Shao,
Deniz Gunduz
Abstract:
3D point cloud is a three-dimensional data format generated by LiDARs and depth sensors, and is being increasingly used in a large variety of applications. This paper presents a novel solution called SEmantic Point cloud Transmission (SEPT), for the transmission of point clouds over wireless channels with limited bandwidth. At the transmitter, SEPT encodes the point cloud via an iterative downsamp…
▽ More
3D point cloud is a three-dimensional data format generated by LiDARs and depth sensors, and is being increasingly used in a large variety of applications. This paper presents a novel solution called SEmantic Point cloud Transmission (SEPT), for the transmission of point clouds over wireless channels with limited bandwidth. At the transmitter, SEPT encodes the point cloud via an iterative downsampling and feature extraction process. At the receiver, SEPT reconstructs the point cloud with latent reconstruction and offset-based upsampling. Extensive numerical experiments confirm that SEPT significantly outperforms the standard approach with octree-based compression followed by channel coding. Compared with a more advanced benchmark that utilizes state-of-the-art deep learning-based compression techniques, SEPT achieves comparable performance while eliminating the cliff and leveling effects. Thanks to its improved performance and robustness against channel variations, we believe that SEPT can be instrumental in collaborative sensing and inference applications among robots and vehicles, particularly in the low-latency and high-mobility scenarios.
△ Less
Submitted 14 June, 2023;
originally announced June 2023.
-
Learning-Based Rich Feedback HARQ for Energy-Efficient Short Packet Transmission
Authors:
Martin Voigt Vejling,
Federico Chiariotti,
Anders Ellersgaard Kalør,
Deniz Gündüz,
Gianluigi Liva,
Petar Popovski
Abstract:
The trade-off between reliability, latency, and energy-efficiency is a central problem in communication systems. Advanced hybrid automated repeat request (HARQ) techniques can reduce the number of retransmissions required for reliable communication, but they have a significant computational cost. On the other hand, strict energy constraints apply mainly to devices, while the access point receiving…
▽ More
The trade-off between reliability, latency, and energy-efficiency is a central problem in communication systems. Advanced hybrid automated repeat request (HARQ) techniques can reduce the number of retransmissions required for reliable communication, but they have a significant computational cost. On the other hand, strict energy constraints apply mainly to devices, while the access point receiving their packets is usually connected to the electrical grid. Therefore, moving the computational complexity required for HARQ schemes from the transmitter to the receiver may provide a way to overcome this trade-off. To achieve this, we propose the Reinforcement-based Adaptive Feedback (RAF) scheme, in which the receiver adaptively learns how much additional redundancy it requires to decode a packet and sends rich feedback (i.e., more than a single bit), requesting the coded retransmission of specific symbols. Simulation results show that the RAF scheme achieves a better trade-off between energy-efficiency, reliability, and latency, compared to existing HARQ solutions and a fixed threshold-based policy. Our RAF scheme can easily adapt to different modulation schemes, and since it relies on the posterior probabilities of the codeword symbols at the decoder, it can generalize to different channel statistics.
△ Less
Submitted 5 June, 2023;
originally announced June 2023.
-
Do not Interfere but Cooperate: A Fully Learnable Code Design for Multi-Access Channels with Feedback
Authors:
Emre Ozfatura,
Chenghong Bian,
Deniz Gunduz
Abstract:
Data-driven deep learning based code designs, including low-complexity neural decoders for existing codes, or end-to-end trainable auto-encoders have exhibited impressive results, particularly in scenarios for which we do not have high-performing structured code designs. However, the vast majority of existing data-driven solutions for channel coding focus on a point-to-point scenario. In this work…
▽ More
Data-driven deep learning based code designs, including low-complexity neural decoders for existing codes, or end-to-end trainable auto-encoders have exhibited impressive results, particularly in scenarios for which we do not have high-performing structured code designs. However, the vast majority of existing data-driven solutions for channel coding focus on a point-to-point scenario. In this work, we consider a multiple access channel (MAC) with feedback and try to understand whether deep learning-based designs are capable of enabling coordination and cooperation among the encoders as well as allowing error correction. Simulation results show that the proposed multi-access block attention feedback (MBAF) code improves the upper bound of the achievable rate of MAC without feedback in finite block length regime.
△ Less
Submitted 1 June, 2023;
originally announced June 2023.
-
Sustainable Edge Intelligence Through Energy-Aware Early Exiting
Authors:
Marcello Bullo,
Seifallah Jardak,
Pietro Carnelli,
Deniz Gündüz
Abstract:
Deep learning (DL) models have emerged as a promising solution for the Internet of Things (IoT). However, due to their computational complexity, DL models consume significant amounts of energy, which can rapidly drain the battery and compromise the performance of IoT devices. For sustainable operation, we consider an edge device with a rechargeable battery and energy harvesting (EH) capabilities.…
▽ More
Deep learning (DL) models have emerged as a promising solution for the Internet of Things (IoT). However, due to their computational complexity, DL models consume significant amounts of energy, which can rapidly drain the battery and compromise the performance of IoT devices. For sustainable operation, we consider an edge device with a rechargeable battery and energy harvesting (EH) capabilities. In addition to the stochastic nature of the ambient energy source, the harvesting rate is often insufficient to meet the inference energy requirements, leading to drastic performance degradation in energy-agnostic devices. To mitigate this problem, we propose energy-adaptive dynamic early exiting (EE) to enable efficient and accurate inference in an EH edge intelligence system. Our approach derives an energy-aware EE policy that determines the optimal amount of computational processing on a per-sample basis. The proposed policy balances the energy consumption to match the limited incoming energy and achieves continuous availability. Numerical results show that accuracy and service rate are improved up to 25% and 35%, respectively, in comparison with an energy-agnostic policy.
△ Less
Submitted 16 July, 2023; v1 submitted 23 May, 2023;
originally announced May 2023.
-
DeepJSCC-l++: Robust and Bandwidth-Adaptive Wireless Image Transmission
Authors:
Chenghong Bian,
Yulin Shao,
Deniz Gunduz
Abstract:
This paper presents a novel vision transformer (ViT) based deep joint source channel coding (DeepJSCC) scheme, dubbed DeepJSCC-l++, which can be adaptive to multiple target bandwidth ratios as well as different channel signal-to-noise ratios (SNRs) using a single model. To achieve this, we train the proposed DeepJSCC-l++ model with different bandwidth ratios and SNRs, which are fed to the model as…
▽ More
This paper presents a novel vision transformer (ViT) based deep joint source channel coding (DeepJSCC) scheme, dubbed DeepJSCC-l++, which can be adaptive to multiple target bandwidth ratios as well as different channel signal-to-noise ratios (SNRs) using a single model. To achieve this, we train the proposed DeepJSCC-l++ model with different bandwidth ratios and SNRs, which are fed to the model as side information. The reconstruction losses corresponding to different bandwidth ratios are calculated, and a new training methodology is proposed, which dynamically assigns different weights to the losses of different bandwidth ratios according to their individual reconstruction qualities. Shifted window (Swin) transformer, is adopted as the backbone for our DeepJSCC-l++ model. Through extensive simulations it is shown that the proposed DeepJSCC-l++ and successive refinement models can adapt to different bandwidth ratios and channel SNRs with marginal performance loss compared to the separately trained models. We also observe the proposed schemes can outperform the digital baseline, which concatenates the BPG compression with capacity-achieving channel code.
△ Less
Submitted 30 November, 2023; v1 submitted 22 May, 2023;
originally announced May 2023.
-
The Rate-Distortion-Perception Trade-off with Side Information
Authors:
Yassine Hamdi,
Deniz Gündüz
Abstract:
In image compression, with recent advances in generative modeling, the existence of a trade-off between the rate and the perceptual quality has been brought to light, where the perception is measured by the closeness of the output distribution to the source. This leads to the question: how does a perception constraint impact the trade-off between the rate and traditional distortion constraints, ty…
▽ More
In image compression, with recent advances in generative modeling, the existence of a trade-off between the rate and the perceptual quality has been brought to light, where the perception is measured by the closeness of the output distribution to the source. This leads to the question: how does a perception constraint impact the trade-off between the rate and traditional distortion constraints, typically quantified by a single-letter distortion measure? We consider the compression of a memoryless source $X$ in the presence of memoryless side information $Z,$ studied by Wyner and Ziv, but elucidate the impact of a perfect realism constraint, which requires the output distribution to match the source distribution. We consider two cases: when $Z$ is available only at the decoder or at both the encoder and the decoder. The rate-distortion trade-off with perfect realism is characterized for sources on general alphabets when infinite common randomness is available between the encoder and the decoder. We show that, similarly to traditional source coding with side information, the two cases are equivalent when $X$ and $Z$ are jointly Gaussian under the squared error distortion measure. We also provide a general inner bound in the case of limited common randomness.
△ Less
Submitted 22 May, 2023;
originally announced May 2023.
-
Unsourced Massive Access-Based Digital Over-the-Air Computation for Efficient Federated Edge Learning
Authors:
Li Qiao,
Zhen Gao,
Zhongxiang Li,
Deniz Gündüz
Abstract:
Over-the-air computation (OAC) is a promising technique to achieve fast model aggregation across multiple devices in federated edge learning (FEEL). In addition to the analog schemes, one-bit digital aggregation (OBDA) scheme was proposed to adapt OAC to modern digital wireless systems. However, one-bit quantization in OBDA can result in a serious information loss and slower convergence of FEEL. T…
▽ More
Over-the-air computation (OAC) is a promising technique to achieve fast model aggregation across multiple devices in federated edge learning (FEEL). In addition to the analog schemes, one-bit digital aggregation (OBDA) scheme was proposed to adapt OAC to modern digital wireless systems. However, one-bit quantization in OBDA can result in a serious information loss and slower convergence of FEEL. To overcome this limitation, this paper proposes an unsourced massive access (UMA)-based generalized digital OAC (GD-OAC) scheme. Specifically, at the transmitter, all the devices share the same non-orthogonal UMA codebook for uplink transmission. The local model update of each device is quantized based on the same quantization codebook. Then, each device transmits a sequence selected from the UMA codebook based on the quantized elements of its model update. At the receiver, we propose an approximate message passing-based algorithm for efficient UMA detection and model aggregation. Simulation results show that the proposed GD-OAC scheme significantly accelerates the FEEL convergences compared with the state-of-the-art OBDA scheme while using the same uplink communication resources.
△ Less
Submitted 17 May, 2023;
originally announced May 2023.
-
Conditional Rate-Distortion-Perception Trade-Off
Authors:
Xueyan Niu,
Deniz Gündüz,
Bo Bai,
Wei Han
Abstract:
Recent advances in machine learning-aided lossy compression are incorporating perceptual fidelity into the rate-distortion theory. In this paper, we study the rate-distortion-perception trade-off when the perceptual quality is measured by the total variation distance between the empirical and product distributions of the discrete memoryless source and its reconstruction. We consider the general se…
▽ More
Recent advances in machine learning-aided lossy compression are incorporating perceptual fidelity into the rate-distortion theory. In this paper, we study the rate-distortion-perception trade-off when the perceptual quality is measured by the total variation distance between the empirical and product distributions of the discrete memoryless source and its reconstruction. We consider the general setting, where two types of resources are available at both the encoder and decoder: a common side information sequence, correlated with the source sequence, and common randomness. We consider both the strong perceptual constraint and the weaker empirical perceptual constraint. The required communication rate for achieving the distortion and empirical perceptual constraint is the minimum conditional mutual information, and similar result holds for strong perceptual constraint when sufficient common randomness is provided and the output along with the side information is constraint to an independent and identically distributed sequence.
△ Less
Submitted 22 May, 2023; v1 submitted 16 May, 2023;
originally announced May 2023.
-
Semantic Communication of Learnable Concepts
Authors:
Francesco Pase,
Szymon Kobus,
Deniz Gunduz,
Michele Zorzi
Abstract:
We consider the problem of communicating a sequence of concepts, i.e., unknown and potentially stochastic maps, which can be observed only through examples, i.e., the mapping rules are unknown. The transmitter applies a learning algorithm to the available examples, and extracts knowledge from the data by optimizing a probability distribution over a set of models, i.e., known functions, which can b…
▽ More
We consider the problem of communicating a sequence of concepts, i.e., unknown and potentially stochastic maps, which can be observed only through examples, i.e., the mapping rules are unknown. The transmitter applies a learning algorithm to the available examples, and extracts knowledge from the data by optimizing a probability distribution over a set of models, i.e., known functions, which can better describe the observed data, and so potentially the underlying concepts. The transmitter then needs to communicate the learned models to a remote receiver through a rate-limited channel, to allow the receiver to decode the models that can describe the underlying sampled concepts as accurately as possible in their semantic space. After motivating our analysis, we propose the formal problem of communicating concepts, and provide its rate-distortion characterization, pointing out its connection with the concepts of empirical and strong coordination in a network. We also provide a bound for the distortion-rate function.
△ Less
Submitted 14 May, 2023;
originally announced May 2023.