Information Theory
See recent articles
Showing new listings for Friday, 27 September 2024
- [1] arXiv:2409.17295 [pdf, html, other]
-
Title: Electromagnetically Consistent Optimization Algorithms for the Global Design of RISComments: Submitted for IEEE publicationSubjects: Information Theory (cs.IT)
The reconfigurable intelligent surface is an emerging technology for wireless communications. We model it as an inhomogeneous boundary of surface impedance, and consider various optimization problems that offer different tradeoffs in terms of performance and implementation complexity. The considered non-convex optimization problems are reformulated as a sequence of approximating linear quadratically constrained or semidefinite programs, which are proved to have a polynomial complexity and to converge monotonically in the objective value.
- [2] arXiv:2409.17546 [pdf, html, other]
-
Title: MASSFormer: Mobility-Aware Spectrum Sensing using Transformer-Driven Tiered StructureSubjects: Information Theory (cs.IT); Machine Learning (cs.LG)
In this paper, we develop a novel mobility-aware transformer-driven tiered structure (MASSFormer) based cooperative spectrum sensing method that effectively models the spatio-temporal dynamics of user movements. Unlike existing methods, our method considers a dynamic scenario involving mobile primary users (PUs) and secondary users (SUs)and addresses the complexities introduced by user mobility. The transformer architecture utilizes an attention mechanism, enabling the proposed method to adeptly model the temporal dynamics of user mobility by effectively capturing long-range dependencies within the input data. The proposed method first computes tokens from the sequence of covariance matrices (CMs) for each SU and processes them in parallel using the SUtransformer network to learn the spatio-temporal features at SUlevel. Subsequently, the collaborative transformer network learns the group-level PU state from all SU-level feature representations. The attention-based sequence pooling method followed by the transformer encoder adjusts the contributions of all tokens. The main goal of predicting the PU states at each SU-level and group-level is to improve detection performance even more. We conducted a sufficient amount of simulations and compared the detection performance of different SS methods. The proposed method is tested under imperfect reporting channel scenarios to show robustness. The efficacy of our method is validated with the simulation results demonstrating its higher performance compared with existing methods in terms of detection probability, sensing error, and classification accuracy.
- [3] arXiv:2409.17553 [pdf, html, other]
-
Title: What Roles can Spatial Modulation and Space Shift Keying Play in LEO Satellite-Assisted Communication?Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
In recent years, the rapid evolution of satellite communications play a pivotal role in addressing the ever-increasing demand for global connectivity, among which the Low Earth Orbit (LEO) satellites attract a great amount of attention due to their low latency and high data throughput capabilities. Based on this, we explore spatial modulation (SM) and space shift keying (SSK) designs as pivotal techniques to enhance spectral efficiency (SE) and bit-error rate (BER) performance in the LEO satellite-assisted multiple-input multiple-output (MIMO) systems. The various performance analysis of these designs are presented in this paper, revealing insightful findings and conclusions through analytical methods and Monte Carlo simulations with perfect and imperfect channel state information (CSI) estimation. The results provide a comprehensive analysis of the merits and trade-offs associated with the investigated schemes, particularly in terms of BER, computational complexity, and SE. This analysis underscores the potential of both schemes as viable candidates for future 6G LEO satellite-assisted wireless communication systems.
- [4] arXiv:2409.17557 [pdf, html, other]
-
Title: Joint Source-Channel Coding: Fundamentals and Recent Progress in Practical DesignsComments: Under review for possible publicationSubjects: Information Theory (cs.IT); Machine Learning (cs.LG)
Semantic- and task-oriented communication has emerged as a promising approach to reducing the latency and bandwidth requirements of next-generation mobile networks by transmitting only the most relevant information needed to complete a specific task at the receiver. This is particularly advantageous for machine-oriented communication of high data rate content, such as images and videos, where the goal is rapid and accurate inference, rather than perfect signal reconstruction. While semantic- and task-oriented compression can be implemented in conventional communication systems, joint source-channel coding (JSCC) offers an alternative end-to-end approach by optimizing compression and channel coding together, or even directly mapping the source signal to the modulated waveform. Although all digital communication systems today rely on separation, thanks to its modularity, JSCC is known to achieve higher performance in finite blocklength scenarios, and to avoid cliff and the levelling-off effects in time-varying channel scenarios. This article provides an overview of the information theoretic foundations of JSCC, surveys practical JSCC designs over the decades, and discusses the reasons for their limited adoption in practical systems. We then examine the recent resurgence of JSCC, driven by the integration of deep learning techniques, particularly through DeepJSCC, highlighting its many surprising advantages in various scenarios. Finally, we discuss why it may be time to reconsider today's strictly separate architectures, and reintroduce JSCC to enable high-fidelity, low-latency communications in critical applications such as autonomous driving, drone surveillance, or wearable systems.
- [5] arXiv:2409.17707 [pdf, html, other]
-
Title: Oversampled Low Ambiguity Zone Sequences for Channel Estimation over Doubly Selective ChannelsSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Pilot sequence design over doubly selective channels (DSC) is challenging due to the variations in both the time- and frequency-domains. Against this background, the contribution of this paper is twofold: Firstly, we investigate the optimal sequence design criteria for efficient channel estimation in orthogonal frequency division multiplexing systems under DSC. Secondly, to design pilot sequences that can satisfy the derived criteria, we propose a new metric called oversampled ambiguity function (O-AF), which considers both fractional and integer Doppler frequency shifts. Optimizing the sidelobes of O-AF through a modified iterative twisted approximation (ITROX) algorithm, we develop a new class of pilot sequences called ``oversampled low ambiguity zone (O-LAZ) sequences". Through numerical experiments, we evaluate the efficiency of the proposed O-LAZ sequences over the traditional low ambiguity zone (LAZ) sequences, Zadoff-Chu (ZC) sequences and m-sequences, by comparing their channel estimation performances over DSC.
- [6] arXiv:2409.17950 [pdf, html, other]
-
Title: An Achievable Rate-Distortion Region for Joint State and Message Communication over Multiple Access ChannelsComments: Accepted by IEEE Information Theory Workshop 2024Subjects: Information Theory (cs.IT)
This paper derives an achievable rate-distortion (R-D) region for the state-dependent discrete memoryless multiple access channel (SD-DMMAC), where the generalized feedback and causal side information are present at encoders, and the decoder performs the joint task of message decoding and state estimation. The Markov coding and backward-forward two-stage decoding schemes are adopted in the proof. This scenario is shown to be capable of modeling various integrated sensing and communication (ISAC) applications, including the monostatic-uplink system and multi-modal sensor networks, which are then studied as examples.
- [7] arXiv:2409.17985 [pdf, html, other]
-
Title: Hypergame Theory for Decentralized Resource Allocation in Multi-user Semantic CommunicationsSubjects: Information Theory (cs.IT); Machine Learning (cs.LG)
Semantic communications (SC) is an emerging communication paradigm in which wireless devices can send only relevant information from a source of data while relying on computing resources to regenerate missing data points. However, the design of a multi-user SC system becomes more challenging because of the computing and communication overhead required for coordination. Existing solutions for learning the semantic language and performing resource allocation often fail to capture the computing and communication tradeoffs involved in multiuser SC. To address this gap, a novel framework for decentralized computing and communication resource allocation in multiuser SC systems is proposed. The challenge of efficiently allocating communication and computing resources (for reasoning) in a decentralized manner to maximize the quality of task experience for the end users is addressed through the application of Stackelberg hyper game theory. Leveraging the concept of second-level hyper games, novel analytical formulations are developed to model misperceptions of the users about each other's communication and control strategies. Further, equilibrium analysis of the learned resource allocation protocols examines the convergence of the computing and communication strategies to a local Stackelberg equilibria, considering misperceptions. Simulation results show that the proposed Stackelberg hyper game results in efficient usage of communication and computing resources while maintaining a high quality of experience for the users compared to state-of-the-art that does not account for the misperceptions.
- [8] arXiv:2409.18094 [pdf, html, other]
-
Title: Mobility in Age-Based Gossip NetworksSubjects: Information Theory (cs.IT); Social and Information Networks (cs.SI); Signal Processing (eess.SP)
We consider a gossiping network where a source forwards updates to a set of $n$ gossiping nodes that are placed in an arbitrary graph structure and gossip with their neighbors. In this paper, we analyze how mobility of nodes affects the freshness of nodes in the gossiping network. To model mobility, we let nodes randomly exchange positions with other nodes in the network. The position of the node determines how the node interacts with the rest of the network. In order to quantify information freshness, we use the version age of information metric. We use the stochastic hybrid system (SHS) framework to derive recursive equations to find the version age for a set of positions in the network in terms of the version ages of sets of positions that are one larger or of the same size. We use these recursive equations to find an upper bound for the average version age of a node in two example networks. We show that mobility can decrease the version age of nodes in a disconnected network from linear scaling in $n$ to at most square root scaling and even to constant scaling in some cases. We perform numerical simulations to analyze how mobility affects the version age of different positions in the network and also show that the upper bounds obtained for the example networks are tight.
New submissions (showing 8 of 8 entries)
- [9] arXiv:2409.17408 (cross-list from cs.CY) [pdf, other]
-
Title: Sociotechnical Approach to Enterprise Generative Artificial Intelligence (E-GenAI)Subjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Information Theory (cs.IT)
In this theoretical article, a sociotechnical approach is proposed to characterize. First, the business ecosystem, focusing on the relationships among Providers, Enterprise, and Customers through SCM, ERP, and CRM platforms to align: (1) Business Intelligence (BI), Fuzzy Logic (FL), and TRIZ (Theory of Inventive Problem Solving), through the OID model, and (2) Knowledge Management (KM) and Imperfect Knowledge Management (IKM), through the OIDK model. Second, the article explores the E-GenAI business ecosystem, which integrates GenAI-based platforms for SCM, ERP, and CRM with GenAI-based platforms for BI, FL, TRIZ, KM, and IKM, to align Large Language Models (LLMs) through the E-GenAI (OID) model. Finally, to understand the dynamics of LLMs, we utilize finite automata to model the relationships between Followers and Followees. This facilitates the construction of LLMs that can identify specific characteristics of users on a social media platform.
- [10] arXiv:2409.17743 (cross-list from quant-ph) [pdf, html, other]
-
Title: Information transmission under Markovian noiseComments: Preliminary version. Comments are welcomeSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT)
We consider an open quantum system undergoing Markovian dynamics, the latter being modelled by a discrete-time quantum Markov semigroup $\{\Phi^n\}_{n \in {\mathbb{N}}}$, resulting from the action of sequential uses of a quantum channel $\Phi$, with $n \in {\mathbb{N}}$ being the discrete time parameter. We find upper and lower bounds on the one-shot $\epsilon$-error information transmission capacities of $\Phi^n$ for a finite time $n\in \mathbb{N}$ and $\epsilon \in [0,1)$ in terms of the structure of the peripheral space of the channel $\Phi$. We consider transmission of $(i)$ classical information (both in the unassisted and entanglement-assisted settings); $(ii)$ quantum information and $(iii)$ private classical information.
Cross submissions (showing 2 of 2 entries)
- [11] arXiv:2404.11881 (replaced) [pdf, html, other]
-
Title: Joint Transmitter and Receiver Design for Movable Antenna Enhanced Multicast CommunicationsComments: 15 pages, 9 figures, accepted by IEEE Transactions on Wireless CommunicationsJournal-ref: IEEE Transactions on Wireless Communications, 2024Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Movable antenna (MA) is an emerging technology that utilizes localized antenna movement to achieve better channel conditions for enhancing communication performance. In this paper, we study the MA-enhanced multicast transmission from a base station equipped with multiple MAs to multiple groups of single-MA users. Our goal is to maximize the minimum weighted signal-to-interference-plus-noise ratio (SINR) among all the users by jointly optimizing the position of each transmit/receive MA and the transmit beamforming. To tackle this challenging problem, we first consider the single-group scenario and propose an efficient algorithm based on the techniques of alternating optimization and successive convex approximation. Particularly, when optimizing transmit or receive MA positions, we construct a concave lower bound for the signal-to-noise ratio (SNR) of each user using only the second-order Taylor expansion, which simplifies the problem-solving process compared to the existing two-step approximation method. The proposed design is then extended to the general multi-group scenario. Simulation results show that the proposed algorithm converges faster than the existing two-step approximation method, achieving a 3.4% enhancement in max-min SNR. Moreover, it can improve the max-min SNR/SINR by up to 22.5%, 181.7%, and 343.9% compared to benchmarks employing only receive MAs, only transmit MAs, and both transmit and receive FPAs, respectively.
- [12] arXiv:2407.21135 (replaced) [pdf, html, other]
-
Title: Physical Modelling and Cancellation of External Passive Intermodulation in FDD MIMOSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
In this paper, the physical approach to model external (air-induced) passive intermodulation (PIM) is presented in a frequency-division duplexing (FDD) multiple-input multiple-output (MIMO) system with an arbitrary number of transceiver chains. The external PIM is a special case of intermodulation distortion (IMD), mainly generated by metallic objects possessing nonlinear properties ("rusty bolt" effect). Typically, such sources are located in the near-field or transition region of the antenna array. PIM products may fall into the receiver band of the FDD system, negatively affecting the uplink signal. In contrast to other works, this one directly simulates the physical external PIM. The system includes models of a point-source external PIM, a finite-length dipole antenna, a MIMO antenna array, and a baseband multicarrier 5G NR OFDM signal. The Channel coefficients method for multi-PIM-source compensation is replicated to verify the proposed external PIM modelling approach. Simulation results of artificially generated PIM cancellation show similar performance as real-life experiments. Therefore, the proposed approach allows testing PIM compensation algorithms on large systems with many antennas and arbitrary array structures. This eliminates the need for experiments with real hardware at the development stage of the PIM cancellation algorithm.
- [13] arXiv:2406.15217 (replaced) [pdf, html, other]
-
Title: Rate-Splitting Multiple Access for Overloaded Multi-group Multicast: A First Experimental StudyComments: Accepted for publication in IEEE Transactions on BroadcastingSubjects: Signal Processing (eess.SP); Information Theory (cs.IT)
Multi-group multicast (MGM) is an increasingly important form of multi-user wireless communications with several potential applications, such as video streaming, federated learning, safety-critical vehicular communications, etc. Rate-Splitting Multiple Access (RSMA) is a powerful interference management technique that can, in principle, achieve higher data rates and greater fairness for all types of multi-user wireless communications, including MGM. This paper presents the first-ever experimental evaluation of RSMA-based MGM, as well as the first-ever three-way comparison of RSMA-based, Space Divison Multiple Access (SDMA)-based and Non-Orthogonal Multiple Access (NOMA)-based MGM. Using a measurement setup involving a two-antenna transmitter and two groups of two single-antenna users per group, we consider the problem of realizing throughput (max-min) fairness across groups for each of three multiple access schemes, over nine experimental cases in a line-of-sight environment capturing varying levels of pathloss difference and channel correlation across the groups. Over these cases, we observe that RSMA-based MGM achieves fairness at a higher throughput for each group than SDMA- and NOMA-based MGM. These findings validate RSMA-based MGM's promised gains from the theoretical literature.
- [14] arXiv:2408.10147 (replaced) [pdf, html, other]
-
Title: In-Context Learning with Representations: Contextual Generalization of Trained TransformersComments: Accepted by NeurIPS 2024Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Information Theory (cs.IT); Optimization and Control (math.OC); Machine Learning (stat.ML)
In-context learning (ICL) refers to a remarkable capability of pretrained large language models, which can learn a new task given a few examples during inference. However, theoretical understanding of ICL is largely under-explored, particularly whether transformers can be trained to generalize to unseen examples in a prompt, which will require the model to acquire contextual knowledge of the prompt for generalization. This paper investigates the training dynamics of transformers by gradient descent through the lens of non-linear regression tasks. The contextual generalization here can be attained via learning the template function for each task in-context, where all template functions lie in a linear space with $m$ basis functions. We analyze the training dynamics of one-layer multi-head transformers to in-contextly predict unlabeled inputs given partially labeled prompts, where the labels contain Gaussian noise and the number of examples in each prompt are not sufficient to determine the template. Under mild assumptions, we show that the training loss for a one-layer multi-head transformer converges linearly to a global minimum. Moreover, the transformer effectively learns to perform ridge regression over the basis functions. To our knowledge, this study is the first provable demonstration that transformers can learn contextual (i.e., template) information to generalize to both unseen examples and tasks when prompts contain only a small number of query-answer pairs.
- [15] arXiv:2409.14264 (replaced) [pdf, html, other]
-
Title: The Differential and Boomerang Properties of a Class of BinomialsSubjects: Number Theory (math.NT); Cryptography and Security (cs.CR); Information Theory (cs.IT)
Let $q$ be an odd prime power with $q\equiv 3\ ({\rm{mod}}\ 4)$. In this paper, we study the differential and boomerang properties of the function $F_{2,u}(x)=x^2\big(1+u\eta(x)\big)$ over $\mathbb{F}_{q}$, where $u\in\mathbb{F}_{q}^*$ and $\eta$ is the quadratic character of $\mathbb{F}_{q}$. We determine the differential uniformity of $F_{2,u}$ for any $u\in\mathbb{F}_{q}^*$ and determine the differential spectra and boomerang uniformity of the locally-APN functions $F_{2,\pm 1}$, thereby disproving a conjecture proposed in \cite{budaghyan2024arithmetization} which states that there exist infinitely many $q$ and $u$ such that $F_{2,u}$ is an APN function.