Showing 1–20 of 20 results for author: Koç, A

Binary Feature Mask Optimization for Feature Selection

Authors: Mehmet E. Lorasdagi, Mehmet Y. Turali, Ali T. Koc, Suleyman S. Kozat

Abstract: We investigate feature selection problem for generic machine learning (ML) models. We introduce a novel framework that selects features considering the predictions of the model. Our framework innovates by using a novel feature masking approach to eliminate the features during the selection process, instead of completely removing them from the dataset. This allows us to use the same ML model during… ▽ More We investigate feature selection problem for generic machine learning (ML) models. We introduce a novel framework that selects features considering the predictions of the model. Our framework innovates by using a novel feature masking approach to eliminate the features during the selection process, instead of completely removing them from the dataset. This allows us to use the same ML model during feature selection, unlike other feature selection methods where we need to train the ML model again as the dataset has different dimensions on each iteration. We obtain the mask operator using the predictions of the ML model, which offers a comprehensive view on the subsets of the features essential for the predictive performance of the model. A variety of approaches exist in the feature selection literature. However, no study has introduced a training-free framework for a generic ML model to select features while considering the importance of the feature subsets as a whole, instead of focusing on the individual features. We demonstrate significant performance improvements on the real-life datasets under different settings using LightGBM and Multi-Layer Perceptron as our ML models. Additionally, we openly share the implementation code for our methods to encourage the research and the contributions in this area. △ Less

Submitted 23 January, 2024; originally announced January 2024.

AFS-BM: Enhancing Model Performance through Adaptive Feature Selection with Binary Masking

Authors: Mehmet Y. Turali, Mehmet E. Lorasdagi, Ali T. Koc, Suleyman S. Kozat

Abstract: We study the problem of feature selection in general machine learning (ML) context, which is one of the most critical subjects in the field. Although, there exist many feature selection methods, however, these methods face challenges such as scalability, managing high-dimensional data, dealing with correlated features, adapting to variable feature importance, and integrating domain knowledge. To t… ▽ More We study the problem of feature selection in general machine learning (ML) context, which is one of the most critical subjects in the field. Although, there exist many feature selection methods, however, these methods face challenges such as scalability, managing high-dimensional data, dealing with correlated features, adapting to variable feature importance, and integrating domain knowledge. To this end, we introduce the "Adaptive Feature Selection with Binary Masking" (AFS-BM) which remedies these problems. AFS-BM achieves this by joint optimization for simultaneous feature selection and model training. In particular, we do the joint optimization and binary masking to continuously adapt the set of features and model parameters during the training process. This approach leads to significant improvements in model accuracy and a reduction in computational requirements. We provide an extensive set of experiments where we compare AFS-BM with the established feature selection methods using well-known datasets from real-life competitions. Our results show that AFS-BM makes significant improvement in terms of accuracy and requires significantly less computational complexity. This is due to AFS-BM's ability to dynamically adjust to the changing importance of features during the training process, which an important contribution to the field. We openly share our code for the replicability of our results and to facilitate further research. △ Less

Submitted 17 June, 2024; v1 submitted 20 January, 2024; originally announced January 2024.

Hierarchical Ensemble-Based Feature Selection for Time Series Forecasting

Authors: Aysin Tumay, Mustafa E. Aydin, Ali T. Koc, Suleyman S. Kozat

Abstract: We introduce a novel ensemble approach for feature selection based on hierarchical stacking for non-stationarity and/or a limited number of samples with a large number of features. Our approach exploits the co-dependency between features using a hierarchical structure. Initially, a machine learning model is trained using a subset of features, and then the output of the model is updated using other… ▽ More We introduce a novel ensemble approach for feature selection based on hierarchical stacking for non-stationarity and/or a limited number of samples with a large number of features. Our approach exploits the co-dependency between features using a hierarchical structure. Initially, a machine learning model is trained using a subset of features, and then the output of the model is updated using other algorithms in a hierarchical manner with the remaining features to minimize the target loss. This hierarchical structure allows for flexible depth and feature selection. By exploiting feature co-dependency hierarchically, our proposed approach overcomes the limitations of traditional feature selection methods and feature importance scores. The effectiveness of the approach is demonstrated on synthetic and well-known real-life datasets, providing significant scalable and stable performance improvements compared to the traditional methods and the state-of-the-art approaches. We also provide the source code of our approach to facilitate further research and replicability of our results. △ Less

Submitted 21 January, 2024; v1 submitted 26 October, 2023; originally announced October 2023.

An Optimistic-Robust Approach for Dynamic Positioning of Omnichannel Inventories

Authors: Pavithra Harsha, Shivaram Subramanian, Ali Koc, Mahesh Ramakrishna, Brian Quanz, Dhruv Shah, Chandra Narayanaswami

Abstract: We introduce a new class of data-driven and distribution-free optimistic-robust bimodal inventory optimization (BIO) strategy to effectively allocate inventory across a retail chain to meet time-varying, uncertain omnichannel demand. While prior Robust optimization (RO) methods emphasize the downside, i.e., worst-case adversarial demand, BIO also considers the upside to remain resilient like RO wh… ▽ More We introduce a new class of data-driven and distribution-free optimistic-robust bimodal inventory optimization (BIO) strategy to effectively allocate inventory across a retail chain to meet time-varying, uncertain omnichannel demand. While prior Robust optimization (RO) methods emphasize the downside, i.e., worst-case adversarial demand, BIO also considers the upside to remain resilient like RO while also reaping the rewards of improved average-case performance by overcoming the presence of endogenous outliers. This bimodal strategy is particularly valuable for balancing the tradeoff between lost sales at the store and the costs of cross-channel e-commerce fulfillment, which is at the core of our inventory optimization model. These factors are asymmetric due to the heterogenous behavior of the channels, with a bias towards the former in terms of lost-sales cost and a dependence on network effects for the latter. We provide structural insights about the BIO solution and how it can be tuned to achieve a preferred tradeoff between robustness and the average-case. Our experiments show that significant benefits can be achieved by rethinking traditional approaches to inventory management, which are siloed by channel and location. Using a real-world dataset from a large American omnichannel retail chain, a business value assessment during a peak period indicates over a 15% profitability gain for BIO over RO and other baselines while also preserving the (practical) worst case performance. △ Less

Submitted 17 October, 2023; originally announced October 2023.

Deep Learning Meets Swarm Intelligence for UAV-Assisted IoT Coverage in Massive MIMO

Authors: Mobeen Mahmood, MohammadMahdi Ghadaksaz, Asil Koc, Tho Le-Ngoc

Abstract: This study considers a UAV-assisted multi-user massive multiple-input multiple-output (MU-mMIMO) systems, where a decode-and-forward (DF) relay in the form of an unmanned aerial vehicle (UAV) facilitates the transmission of multiple data streams from a base station (BS) to multiple Internet-of-Things (IoT) users. A joint optimization problem of hybrid beamforming (HBF), UAV relay positioning, and… ▽ More This study considers a UAV-assisted multi-user massive multiple-input multiple-output (MU-mMIMO) systems, where a decode-and-forward (DF) relay in the form of an unmanned aerial vehicle (UAV) facilitates the transmission of multiple data streams from a base station (BS) to multiple Internet-of-Things (IoT) users. A joint optimization problem of hybrid beamforming (HBF), UAV relay positioning, and power allocation (PA) to multiple IoT users to maximize the total achievable rate (AR) is investigated. The study adopts a geometry-based millimeter-wave (mmWave) channel model for both links and proposes three different swarm intelligence (SI)-based algorithmic solutions to optimize: 1) UAV location with equal PA; 2) PA with fixed UAV location; and 3) joint PA with UAV deployment. The radio frequency (RF) stages are designed to reduce the number of RF chains based on the slow time-varying angular information, while the baseband (BB) stages are designed using the reduced-dimension effective channel matrices. Then, a novel deep learning (DL)-based low-complexity joint hybrid beamforming, UAV location and power allocation optimization scheme (J-HBF-DLLPA) is proposed via fully-connected deep neural network (DNN), consisting of an offline training phase, and an online prediction of UAV location and optimal power values for maximizing the AR. The illustrative results show that the proposed algorithmic solutions can attain higher capacity and reduce average delay for delay-constrained transmissions in a UAV-assisted MU-mMIMO IoT systems. Additionally, the proposed J-HBF-DLLPA can closely approach the optimal capacity while significantly reducing the runtime by 99%, which makes the DL-based solution a promising implementation for real-time online applications in UAV-assisted MU-mMIMO IoT systems. △ Less

Submitted 20 September, 2023; originally announced September 2023.

Sub-Array Selection in Full-Duplex Massive MIMO for Enhanced Self-Interference Suppression

Authors: Mobeen Mahmood, Asil Koc, Duc Tuong Nguyen, Robert Morawski, Tho Le-Ngoc

Abstract: This study considers a novel full-duplex (FD) massive multiple-input multiple-output (mMIMO) system using hybrid beamforming (HBF) architecture, which allows for simultaneous uplink (UL) and downlink (DL) transmission over the same frequency band. Particularly, our objective is to mitigate the strong self-interference (SI) solely on the design of UL and DL RF beamforming stages jointly with sub-ar… ▽ More This study considers a novel full-duplex (FD) massive multiple-input multiple-output (mMIMO) system using hybrid beamforming (HBF) architecture, which allows for simultaneous uplink (UL) and downlink (DL) transmission over the same frequency band. Particularly, our objective is to mitigate the strong self-interference (SI) solely on the design of UL and DL RF beamforming stages jointly with sub-array selection (SAS) for transmit (Tx) and receive (Rx) sub-arrays at base station (BS). Based on the measured SI channel in an anechoic chamber, we propose a min-SI beamforming scheme with SAS, which applies perturbations to the beam directivity to enhance SI suppression in UL and DL beam directions. To solve this challenging nonconvex optimization problem, we propose a swarm intelligence-based algorithmic solution to find the optimal perturbations as well as the Tx and Rx sub-arrays to minimize SI subject to the directivity degradation constraints for the UL and DL beams. The results show that the proposed min-SI BF scheme can achieve SI suppression as high as 78 dB in FD mMIMO systems. △ Less

Submitted 6 September, 2023; originally announced September 2023.

Comments: This paper has been accepted for publication in IEEE Globecom 2023

Fast-FNet: Accelerating Transformer Encoder Models via Efficient Fourier Layers

Authors: Nurullah Sevim, Ege Ozan Özyedek, Furkan Şahinuç, Aykut Koç

Abstract: Transformer-based language models utilize the attention mechanism for substantial performance improvements in almost all natural language processing (NLP) tasks. Similar attention structures are also extensively studied in several other areas. Although the attention mechanism enhances the model performances significantly, its quadratic complexity prevents efficient processing of long sequences. Re… ▽ More Transformer-based language models utilize the attention mechanism for substantial performance improvements in almost all natural language processing (NLP) tasks. Similar attention structures are also extensively studied in several other areas. Although the attention mechanism enhances the model performances significantly, its quadratic complexity prevents efficient processing of long sequences. Recent works focused on eliminating the disadvantages of computational inefficiency and showed that transformer-based models can still reach competitive results without the attention layer. A pioneering study proposed the FNet, which replaces the attention layer with the Fourier Transform (FT) in the transformer encoder architecture. FNet achieves competitive performances concerning the original transformer encoder model while accelerating training process by removing the computational burden of the attention mechanism. However, the FNet model ignores essential properties of the FT from the classical signal processing that can be leveraged to increase model efficiency further. We propose different methods to deploy FT efficiently in transformer encoder models. Our proposed architectures have smaller number of model parameters, shorter training times, less memory usage, and some additional performance improvements. We demonstrate these improvements through extensive experiments on common benchmarks. △ Less

Submitted 16 May, 2023; v1 submitted 26 September, 2022; originally announced September 2022.

Comments: 11 pages

Unsupervised Simplification of Legal Texts

Authors: Mert Cemri, Tolga Çukur, Aykut Koç

Abstract: The processing of legal texts has been developing as an emerging field in natural language processing (NLP). Legal texts contain unique jargon and complex linguistic attributes in vocabulary, semantics, syntax, and morphology. Therefore, the development of text simplification (TS) methods specific to the legal domain is of paramount importance for facilitating comprehension of legal text by ordina… ▽ More The processing of legal texts has been developing as an emerging field in natural language processing (NLP). Legal texts contain unique jargon and complex linguistic attributes in vocabulary, semantics, syntax, and morphology. Therefore, the development of text simplification (TS) methods specific to the legal domain is of paramount importance for facilitating comprehension of legal text by ordinary people and providing inputs to high-level models for mainstream legal NLP applications. While a recent study proposed a rule-based TS method for legal text, learning-based TS in the legal domain has not been considered previously. Here we introduce an unsupervised simplification method for legal texts (USLT). USLT performs domain-specific TS by replacing complex words and splitting long sentences. To this end, USLT detects complex words in a sentence, generates candidates via a masked-transformer model, and selects a candidate for substitution based on a rank score. Afterward, USLT recursively decomposes long sentences into a hierarchy of shorter core and context sentences while preserving semantic meaning. We demonstrate that USLT outperforms state-of-the-art domain-general TS methods in text simplicity while keeping the semantics intact. △ Less

Submitted 1 September, 2022; originally announced September 2022.

RIS-Aided Angular-Based Hybrid Beamforming Design in mmWave Massive MIMO Systems

Authors: Ibrahim Yildirim, Asil Koc, Ertugrul Basar, Tho Le-Ngoc

Abstract: This paper proposes a reconfigurable intelligent surface (RIS)-aided and angular-based hybrid beamforming (AB-HBF) technique for the millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems. The proposed RIS-AB-HBF architecture consists of three stages: (i) RF beamformer, (ii) baseband (BB) precoder/combiner, and (iii) RIS phase shift design. First, in order to reduce the num… ▽ More This paper proposes a reconfigurable intelligent surface (RIS)-aided and angular-based hybrid beamforming (AB-HBF) technique for the millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems. The proposed RIS-AB-HBF architecture consists of three stages: (i) RF beamformer, (ii) baseband (BB) precoder/combiner, and (iii) RIS phase shift design. First, in order to reduce the number of RF chains and the channel estimation overhead, RF beamformers are designed based on the 3D geometry-based mmWave channel model using slow time-varying angular parameters of the channel. Second, a BB precoder/combiner is designed by exploiting the reduced-size effective channel seen from the BB stages. Then, the phase shifts of the RIS are adjusted to maximize the achievable rate of the system via the nature-inspired particle swarm optimization (PSO) algorithm. Illustrative simulation results demonstrate that the use of RISs in the AB-HBF systems has the potential to provide more promising advantages in terms of reliability and flexibility in system design. △ Less

Submitted 13 August, 2022; originally announced August 2022.

Comments: Accepted for presentation at the IEEE GLOBECOM 2022

Nature-Inspired Intelligent α-Fair Hybrid Precoding in Multiuser Massive Multiple-Input Multiple-Output Systems

Authors: Asil Koc, Tho Le-Ngoc

Abstract: This paper proposes a novel nature-inspired $α$-fair hybrid precoding (NI-$α$HP) technique for millimeter-wave multi-user massive multiple-input multiple-output systems. Unlike the existing HP literature, we propose to apply $α$-fairness for maintaining various fairness expectations (e.g., sum-rate maximization, proportional fairness, max-min fairness, etc.). After developing the analog RF beamfor… ▽ More This paper proposes a novel nature-inspired $α$-fair hybrid precoding (NI-$α$HP) technique for millimeter-wave multi-user massive multiple-input multiple-output systems. Unlike the existing HP literature, we propose to apply $α$-fairness for maintaining various fairness expectations (e.g., sum-rate maximization, proportional fairness, max-min fairness, etc.). After developing the analog RF beamformer via slow time-varying angular information, the digital baseband (BB) precoder is designed via the reduced-dimensional effective channel matrix seen from the BB-stage. For the $α$-fairness, we derive the optimal digital BB precoder expression with a set of parameters, where optimizing them is an NP-hard problem. Hence, we efficiently optimize the parameters in the digital BB precoder via five nature-inspired intelligent algorithms. Numerical results present that when the sum-rate maximization is the target, the proposed NI-$α$HP technique greatly improves the sum-rate capacity and energy-efficiency performance compared to other benchmarks. Moreover, NI-$α$HP supports different fairness expectations and reduces the rate gap among UEs by varying the fairness level ($α$). △ Less

Submitted 18 July, 2022; originally announced July 2022.

Comments: 15 pages, 7 figures

Joint Time-Vertex Fractional Fourier Transform

Authors: Tuna Alikaşifoğlu, Bünyamin Kartal, Eray Özgünay, Aykut Koç

Abstract: Graph signal processing (GSP) facilitates the analysis of high-dimensional data on non-Euclidean domains by utilizing graph signals defined on graph vertices. In addition to static data, each vertex can provide continuous time-series signals, transforming graph signals into time-series signals on each vertex. The joint time-vertex Fourier transform (JFT) framework offers spectral analysis capabili… ▽ More Graph signal processing (GSP) facilitates the analysis of high-dimensional data on non-Euclidean domains by utilizing graph signals defined on graph vertices. In addition to static data, each vertex can provide continuous time-series signals, transforming graph signals into time-series signals on each vertex. The joint time-vertex Fourier transform (JFT) framework offers spectral analysis capabilities to analyze these joint time-vertex signals. Analogous to the fractional Fourier transform (FRT) extending the ordinary Fourier transform (FT), we introduce the joint time-vertex fractional Fourier transform (JFRT) as a generalization of JFT. The JFRT enables fractional analysis for joint time-vertex processing by extending Fourier analysis to fractional orders in both temporal and vertex domains. We theoretically demonstrate that JFRT generalizes JFT and maintains properties such as index additivity, reversibility, reduction to identity, and unitarity for specific graph topologies. Additionally, we derive Tikhonov regularization-based denoising in the JFRT domain, ensuring robust and well-behaved solutions. Comprehensive numerical experiments on synthetic and real-world datasets highlight the effectiveness of JFRT in denoising and clustering tasks that outperform state-of-the-art approaches. △ Less

Submitted 10 July, 2024; v1 submitted 15 March, 2022; originally announced March 2022.

Comments: 30 pages, 8 figures

Energy-Efficient Throughput Maximization in mmWave MU-Massive-MIMO-OFDM: Genetic Algorithm based Resource Allocation

Authors: Asil Koc, Farhan Bishe, Tho Le-Ngoc

Abstract: This paper develops a new genetic algorithm based resource allocation (GA-RA) technique for energy-efficient throughout maximization in multi-user massive multiple-input multiple-output (MU-mMIMO) systems using orthogonal frequency division multiplexing (OFDM) based transmission. We employ a hybrid precoding (HP) architecture with three stages: (i) radio frequency (RF) beamformer, (ii) baseband (B… ▽ More This paper develops a new genetic algorithm based resource allocation (GA-RA) technique for energy-efficient throughout maximization in multi-user massive multiple-input multiple-output (MU-mMIMO) systems using orthogonal frequency division multiplexing (OFDM) based transmission. We employ a hybrid precoding (HP) architecture with three stages: (i) radio frequency (RF) beamformer, (ii) baseband (BB) precoder, (iii) resource allocation (RA) block. First, a single RF beamformer block is built for all subcarriers via the slow time-varying angle-of-departure (AoD) information. For enhancing the energy efficiency, the RF beamformer aims to reduce the hardware cost/complexity and total power consumption via a low number of RF chains. Afterwards, the reduced-size effective channel state information (CSI) is utilized in the design of a distinct BB precoder and RA block for each subcarrier. The BB precoder is developed via regularized zero-forcing technique. Finally, the RA block is built via the proposed GA-RA technique for throughput maximization by allocating the power and subcarrier resources. The illustrative results show that the throughput performance in the MU-mMIMO-OFDM systems is greatly enhanced via the proposed GA-RA technique compared to both equal RA (EQ-RA) and particle swarm optimization based RA (PSO-RA). Moreover, the performance gain ratio increases with the increasing number of subcarriers, particularly for low transmission powers. △ Less

Submitted 18 February, 2022; originally announced February 2022.

Comments: 6 pages, 4 figures, conference

A Deep Learning and Geospatial Data-Based Channel Estimation Technique for Hybrid Massive MIMO Systems

Authors: Xiaoyi Zhu, Asil Koc, Robert Morawski, Tho Le-Ngoc

Abstract: This paper presents a novel channel estimation technique for the multi-user massive multiple-input multiple-output (MU-mMIMO) systems using angular-based hybrid precoding (AB-HP). The proposed channel estimation technique generates group-wise channel state information (CSI) of user terminal (UT) zones in the service area by deep neural networks (DNN) and fuzzy c-Means (FCM) clustering. The slow ti… ▽ More This paper presents a novel channel estimation technique for the multi-user massive multiple-input multiple-output (MU-mMIMO) systems using angular-based hybrid precoding (AB-HP). The proposed channel estimation technique generates group-wise channel state information (CSI) of user terminal (UT) zones in the service area by deep neural networks (DNN) and fuzzy c-Means (FCM) clustering. The slow time-varying CSI between the base station (BS) and feasible UT locations in the service area is calculated from the geospatial data by offline ray tracing and a DNN-based path estimation model associated with the 1-dimensional convolutional neural network (1D-CNN) and regression tree ensembles. Then, the UT-level CSI of all feasible locations is grouped into clusters by a proposed FCM clustering. Finally, the service area is divided into a number of non-overlapping UT zones. Each UT zone is characterized by a corresponding set of clusters named as UT-group CSI, which is utilized in the analog RF beamformer design of AB-HP to reduce the required large online CSI overhead in the MU-mMIMO systems. Then, the reduced-size online CSI is employed in the baseband (BB) precoder of AB-HP. Simulations are conducted in the indoor scenario at 28 GHz and tested in an AB-HP MU-mMIMO system with a uniform rectangular array (URA) having 16x16=256 antennas and 22 RF chains. Illustrative results indicate that 91.4% online CSI can be reduced by using the proposed offline channel estimation technique as compared to the conventional online channel sounding. The proposed DNN-based path estimation technique produces same amount of UT-level CSI with runtime reduced by 65.8% as compared to the computationally expensive ray tracing. △ Less

Submitted 29 January, 2022; originally announced January 2022.

Comments: 18 pages, 21 figures

Full-Duplex Non-Coherent Communications for Massive MIMO Systems with Analog Beamforming

Authors: Asil Koc, Ahmed Masmoudi, Tho Le-Ngoc

Abstract: In this paper, a novel full-duplex non-coherent (FD-NC) transmission scheme is developed for massive multiple-input multiple-output (mMIMO) systems using analog beamforming (ABF). We propose to use a structured Grassmannian constellation for the non-coherent communications that does not require channel estimation. Then, we design the transmit and receive ABF via the slow time-varying angle-of-depa… ▽ More In this paper, a novel full-duplex non-coherent (FD-NC) transmission scheme is developed for massive multiple-input multiple-output (mMIMO) systems using analog beamforming (ABF). We propose to use a structured Grassmannian constellation for the non-coherent communications that does not require channel estimation. Then, we design the transmit and receive ABF via the slow time-varying angle-of-departure (AoD) and angle-of-arrival (AoA) information, respectively. The ABF design targets maximizing the intended signal power while suppressing the strong self-interference (SI) occurred in the FD transmission. Also, the proposed ABF technique only needs a single transmit and receive RF chain to support large antenna arrays, thus, it reduces hardware cost/complexity in the mMIMO systems. It is shown that the proposed FD-NC offers a great improvement in bit error rate (BER) in comparison to both half-duplex non-coherent (HD-NC) and HD coherent schemes. We also observe that the proposed FD-NC both reduces the error floor resulted from the residual SI in FD transmission, and provides lower BER compared to the FD coherent transmission. △ Less

Submitted 29 January, 2022; originally announced January 2022.

Comments: 6 pages, 6 figures

Deep Learning based Multi-User Power Allocation and Hybrid Precoding in Massive MIMO Systems

Authors: Asil Koc, Mike Wang, Tho Le-Ngoc

Abstract: This paper proposes a deep learning based power allocation (DL-PA) and hybrid precoding technique for multiuser massive multiple-input multiple-output (MU-mMIMO) systems. We first utilize an angular-based hybrid precoding technique for reducing the number of RF chains and channel estimation overhead. Then, we develop the DL-PA algorithm via a fully-connected deep neural network (DNN). DL-PA has tw… ▽ More This paper proposes a deep learning based power allocation (DL-PA) and hybrid precoding technique for multiuser massive multiple-input multiple-output (MU-mMIMO) systems. We first utilize an angular-based hybrid precoding technique for reducing the number of RF chains and channel estimation overhead. Then, we develop the DL-PA algorithm via a fully-connected deep neural network (DNN). DL-PA has two phases: (i) offline supervised learning with the optimal allocated powers obtained by particle swarm optimization based PA (PSO-PA) algorithm, (ii) online power prediction by the trained DNN. In comparison to the computationally expensive PSO-PA, it is shown that DL-PA greatly reduces the runtime by 98.6%-99.9%, while closely achieving the optimal sum-rate capacity. It makes DL-PA a promising algorithm for the real-time online applications in MU-mMIMO systems. △ Less

Submitted 29 January, 2022; originally announced January 2022.

Comments: 6 pages, 6 figures

Multi-Label Sentiment Analysis on 100 Languages with Dynamic Weighting for Label Imbalance

Authors: Selim F. Yilmaz, E. Batuhan Kaynak, Aykut Koç, Hamdi Dibeklioğlu, Suleyman S. Kozat

Abstract: We investigate cross-lingual sentiment analysis, which has attracted significant attention due to its applications in various areas including market research, politics and social sciences. In particular, we introduce a sentiment analysis framework in multi-label setting as it obeys Plutchik wheel of emotions. We introduce a novel dynamic weighting method that balances the contribution from each cl… ▽ More We investigate cross-lingual sentiment analysis, which has attracted significant attention due to its applications in various areas including market research, politics and social sciences. In particular, we introduce a sentiment analysis framework in multi-label setting as it obeys Plutchik wheel of emotions. We introduce a novel dynamic weighting method that balances the contribution from each class during training, unlike previous static weighting methods that assign non-changing weights based on their class frequency. Moreover, we adapt the focal loss that favors harder instances from single-label object recognition literature to our multi-label setting. Furthermore, we derive a method to choose optimal class-specific thresholds that maximize the macro-f1 score in linear time complexity. Through an extensive set of experiments, we show that our method obtains the state-of-the-art performance in 7 of 9 metrics in 3 different languages using a single model compared to the common baselines and the best-performing methods in the SemEval competition. We publicly share our code for our model, which can perform sentiment analysis in 100 languages, to facilitate further research. △ Less

Submitted 26 August, 2020; originally announced August 2020.

Comments: 11 pages, 6 figures

Quadruplet Selection Methods for Deep Embedding Learning

Authors: Kaan Karaman, Erhan Gundogdu, Aykut Koc, A. Aydin Alatan

Abstract: Recognition of objects with subtle differences has been used in many practical applications, such as car model recognition and maritime vessel identification. For discrimination of the objects in fine-grained detail, we focus on deep embedding learning by using a multi-task learning framework, in which the hierarchical labels (coarse and fine labels) of the samples are utilized both for classifica… ▽ More Recognition of objects with subtle differences has been used in many practical applications, such as car model recognition and maritime vessel identification. For discrimination of the objects in fine-grained detail, we focus on deep embedding learning by using a multi-task learning framework, in which the hierarchical labels (coarse and fine labels) of the samples are utilized both for classification and a quadruplet-based loss function. In order to improve the recognition strength of the learned features, we present a novel feature selection method specifically designed for four training samples of a quadruplet. By experiments, it is observed that the selection of very hard negative samples with relatively easy positive ones from the same coarse and fine classes significantly increases some performance metrics in a fine-grained dataset when compared to selecting the quadruplet samples randomly. The feature embedding learned by the proposed method achieves favorable performance against its state-of-the-art counterparts. △ Less

Submitted 22 July, 2019; originally announced July 2019.

Comments: 6 pages, 2 figures, accepted by IEEE ICIP 2019

Deep Iterative Reconstruction for Phase Retrieval

Authors: Çağatay Işıl, Figen S. Oktem, Aykut Koç

Abstract: Classical phase retrieval problem is the recovery of a constrained image from the magnitude of its Fourier transform. Although there are several well-known phase retrieval algorithms including the hybrid input-output (HIO) method, the reconstruction performance is generally sensitive to initialization and measurement noise. Recently, deep neural networks (DNNs) have been shown to provide state-of-… ▽ More Classical phase retrieval problem is the recovery of a constrained image from the magnitude of its Fourier transform. Although there are several well-known phase retrieval algorithms including the hybrid input-output (HIO) method, the reconstruction performance is generally sensitive to initialization and measurement noise. Recently, deep neural networks (DNNs) have been shown to provide state-of-the-art performance in solving several inverse problems such as denoising, deconvolution, and superresolution. In this work, we develop a phase retrieval algorithm that utilizes two DNNs together with the model-based HIO method. First, a DNN is trained to remove the HIO artifacts and is used iteratively with the HIO method to improve the reconstructions. After this iterative phase, a second DNN is trained to remove the remaining artifacts. Numerical results demonstrate the effectiveness of ourapproach, which has little additional computational cost compared to the HIO method. Our approach not only achieves state-of-the-art reconstruction performance but also is more robust to different initialization and noise levels. △ Less

Submitted 19 August, 2019; v1 submitted 25 April, 2019; originally announced April 2019.

Comments: 14 pages, 8 figures, published in Applied Optics (Vol. 58, Issue 20, pp. 5422-5431 (2019))

Journal ref: Çağatay Işıl, Figen S. Oktem, and Aykut Koç, "Deep iterative reconstruction for phase retrieval," Appl. Opt. 58, 5422-5431 (2019)

Imparting Interpretability to Word Embeddings while Preserving Semantic Structure

Authors: Lutfi Kerem Senel, Ihsan Utlu, Furkan Şahinuç, Haldun M. Ozaktas, Aykut Koç

Abstract: As an ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They capture semantic and syntactic relations among words but the vectors corresponding to the words are only meaningful relative to each other. Neither the vector nor its dimensions have any absolute, interpretable meaning. We int… ▽ More As an ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They capture semantic and syntactic relations among words but the vectors corresponding to the words are only meaningful relative to each other. Neither the vector nor its dimensions have any absolute, interpretable meaning. We introduce an additive modification to the objective function of the embedding learning algorithm that encourages the embedding vectors of words that are semantically related to a predefined concept to take larger values along a specified dimension, while leaving the original semantic learning mechanism mostly unaffected. In other words, we align words that are already determined to be related, along predefined concepts. Therefore, we impart interpretability to the word embedding by assigning meaning to its vector dimensions. The predefined concepts are derived from an external lexical resource, which in this paper is chosen as Roget's Thesaurus. We observe that alignment along the chosen concepts is not limited to words in the Thesaurus and extends to other related words as well. We quantify the extent of interpretability and assignment of meaning from our experimental results. Manual human evaluation results have also been presented to further verify that the proposed method increases interpretability. We also demonstrate the preservation of semantic coherence of the resulting vector space by using word-analogy and word-similarity tests. These tests show that the interpretability-imparted word embeddings that are obtained by the proposed framework do not sacrifice performances in common benchmark tests. △ Less

Submitted 2 July, 2020; v1 submitted 19 July, 2018; originally announced July 2018.

Comments: 14 pages, 5 figures

Journal ref: Natural Language Engineering, 1-26, 2020

Semantic Structure and Interpretability of Word Embeddings

Authors: Lutfi Kerem Senel, Ihsan Utlu, Veysel Yucesoy, Aykut Koc, Tolga Cukur

Abstract: Dense word embeddings, which encode semantic meanings of words to low dimensional vector spaces have become very popular in natural language processing (NLP) research due to their state-of-the-art performances in many NLP tasks. Word embeddings are substantially successful in capturing semantic relations among words, so a meaningful semantic structure must be present in the respective vector space… ▽ More Dense word embeddings, which encode semantic meanings of words to low dimensional vector spaces have become very popular in natural language processing (NLP) research due to their state-of-the-art performances in many NLP tasks. Word embeddings are substantially successful in capturing semantic relations among words, so a meaningful semantic structure must be present in the respective vector spaces. However, in many cases, this semantic structure is broadly and heterogeneously distributed across the embedding dimensions, which makes interpretation a big challenge. In this study, we propose a statistical method to uncover the latent semantic structure in the dense word embeddings. To perform our analysis we introduce a new dataset (SEMCAT) that contains more than 6500 words semantically grouped under 110 categories. We further propose a method to quantify the interpretability of the word embeddings; the proposed method is a practical alternative to the classical word intrusion test that requires human intervention. △ Less

Submitted 16 May, 2018; v1 submitted 1 November, 2017; originally announced November 2017.

Comments: 11 Pages, 8 Figures, accepted by IEEE/ACM Transactions on Audio, Speech, and Language Processing

Journal ref: L. K. Şenel, İ. Utlu, V. Yücesoy, A. Koç and T. Çukur, "Semantic Structure and Interpretability of Word Embeddings," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 10, pp. 1769-1779, Oct. 2018