-
A Hybrid Deep Learning Classification of Perimetric Glaucoma Using Peripapillary Nerve Fiber Layer Reflectance and Other OCT Parameters from Three Anatomy Regions
Authors:
Ou Tan,
David S. Greenfield,
Brian A. Francis,
Rohit Varma,
Joel S. Schuman,
David Huang,
Dongseok Choi
Abstract:
Precis: A hybrid deep-learning model combines NFL reflectance and other OCT parameters to improve glaucoma diagnosis. Objective: To investigate if a deep learning model could be used to combine nerve fiber layer (NFL) reflectance and other OCT parameters for glaucoma diagnosis. Patients and Methods: This is a prospective observational study where of 106 normal subjects and 164 perimetric glaucoma…
▽ More
Precis: A hybrid deep-learning model combines NFL reflectance and other OCT parameters to improve glaucoma diagnosis. Objective: To investigate if a deep learning model could be used to combine nerve fiber layer (NFL) reflectance and other OCT parameters for glaucoma diagnosis. Patients and Methods: This is a prospective observational study where of 106 normal subjects and 164 perimetric glaucoma (PG) patients. Peripapillary NFL reflectance map, NFL thickness map, optic head analysis of disc, and macular ganglion cell complex thickness were obtained using spectral domain OCT. A hybrid deep learning model combined a fully connected network (FCN) and a convolution neural network (CNN) to develop and combine those OCT maps and parameters to distinguish normal and PG eyes. Two deep learning models were compared based on whether the NFL reflectance map was used as part of the input or not. Results: The hybrid deep learning model with reflectance achieved 0.909 sensitivity at 99% specificity and 0.926 at 95%. The overall accuracy was 0.948 with 0.893 sensitivity and 1.000 specificity, and the AROC was 0.979, which is significantly better than the logistic regression models (p < 0.001). The second best model is the hybrid deep learning model w/o reflectance, which also had significantly higher AROC than logistic regression models (p < 0.001). Logistic regression with reflectance model had slightly higher AROC or sensitivity than the other logistic regression model without reflectance (p = 0.024). Conclusions: Hybrid deep learning model significantly improved the diagnostic accuracy, without or without NFL reflectance. Hybrid deep learning model, combining reflectance/NFL thickness/GCC thickness/ONH parameter, may be a practical model for glaucoma screen purposes.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
Input-length-shortening and text generation via attention values
Authors:
Neşet Özkan Tan,
Alex Yuxuan Peng,
Joshua Bensemann,
Qiming Bao,
Tim Hartill,
Mark Gahegan,
Michael Witbrock
Abstract:
Identifying words that impact a task's performance more than others is a challenge in natural language processing. Transformers models have recently addressed this issue by incorporating an attention mechanism that assigns greater attention (i.e., relevance) scores to some words than others. Because of the attention mechanism's high computational cost, transformer models usually have an input-leng…
▽ More
Identifying words that impact a task's performance more than others is a challenge in natural language processing. Transformers models have recently addressed this issue by incorporating an attention mechanism that assigns greater attention (i.e., relevance) scores to some words than others. Because of the attention mechanism's high computational cost, transformer models usually have an input-length limitation caused by hardware constraints. This limitation applies to many transformers, including the well-known bidirectional encoder representations of the transformer (BERT) model. In this paper, we examined BERT's attention assignment mechanism, focusing on two questions: (1) How can attention be employed to reduce input length? (2) How can attention be used as a control mechanism for conditional text generation? We investigated these questions in the context of a text classification task. We discovered that BERT's early layers assign more critical attention scores for text classification tasks compared to later layers. We demonstrated that the first layer's attention sums could be used to filter tokens in a given sequence, considerably decreasing the input length while maintaining good test accuracy. We also applied filtering, which uses a compute-efficient semantic similarities algorithm, and discovered that retaining approximately 6\% of the original sequence is sufficient to obtain 86.5\% accuracy. Finally, we showed that we could generate data in a stable manner and indistinguishable from the original one by only using a small percentage (10\%) of the tokens with high attention scores according to BERT's first layer.
△ Less
Submitted 13 March, 2023;
originally announced March 2023.
-
Scientific Impact of Graph-Based Approaches in Deep Learning Studies -- A Bibliometric Comparison
Authors:
Ilker Turker,
Serhat Orkun Tan
Abstract:
Applying graph-based approaches in deep learning receives more attention over time. This study presents statistical analysis on the use of graph-based approaches in deep learning and examines the scientific impact of the related articles. Processing the data obtained from the Web of Science database, metrics such as the type of the articles, funding availability, indexing type, annual average numb…
▽ More
Applying graph-based approaches in deep learning receives more attention over time. This study presents statistical analysis on the use of graph-based approaches in deep learning and examines the scientific impact of the related articles. Processing the data obtained from the Web of Science database, metrics such as the type of the articles, funding availability, indexing type, annual average number of citations and the number of access were analyzed to quantitatively reveal the effects on the scientific audience. It's outlined that deep learning-based studies gained momentum after year 2013, and the rate of graph-based approaches in all deep learning studies increased linearly from 1% to 4% within the following 10 years. Conference publications scanned in the Conference Proceeding Citation Index (CPCI) on the graph-based approaches receive significantly more citations. The citation counts of the SCI-Expanded and Emerging SCI indexed publications of the two streams are close to each other. While the citation performances of the supported and unsupported publications of the two sides were similar, pure deep learning studies received more citations on the journal publication side and graph-based approaches received more citations on the conference side. Despite their similar performance in recent years, graph-based studies show twice more citation performance as they get older, compared to traditional approaches. Annual average citation performance per article for all deep learning studies is 11.051 in 2014, while it is 22.483 for graph-based studies. Also, despite receiving 16% more access, graph-based papers get almost the same overall citation over time with the pure counterpart. This is an indication that graph-based approaches need a greater bunch of attention to follow, while pure deep learning counterpart is relatively simpler to get inside.
△ Less
Submitted 13 October, 2022;
originally announced October 2022.
-
Machine Learning vs. Deep Learning in 5G Networks -- A Comparison of Scientific Impact
Authors:
Ilker Turker,
Serhat Orkun Tan
Abstract:
Introduction of fifth generation (5G) wireless network technology has matched the crucial need for high capacity and speed needs of the new generation mobile applications. Recent advances in Artificial Intelligence (AI) also empowered 5G cellular networks with two mainstreams as machine learning (ML) and deep learning (DL) techniques. Our study aims to uncover the differences in scientific impact…
▽ More
Introduction of fifth generation (5G) wireless network technology has matched the crucial need for high capacity and speed needs of the new generation mobile applications. Recent advances in Artificial Intelligence (AI) also empowered 5G cellular networks with two mainstreams as machine learning (ML) and deep learning (DL) techniques. Our study aims to uncover the differences in scientific impact for these two techniques by the means of statistical bibliometrics. The performed analysis includes citation performance with respect to indexing types, funding availability, journal or conference publishing options together with distributions of these metrics along years to evaluate the popularity trends in a detailed manner. Web of Science (WoS) database host 2245 papers for ML and 1407 papers for DL-related studies. DL studies, starting with 9% rate in 2013, has reached to 45% rate in 2022 among all DL and ML-related studies. Results related to scientific impact indicate that DL studies get slightly more average normalized citation (2.256) compared to ML studies (2.118) in 5G, while SCI-Expanded indexed papers in both sides tend to have similar citation performance (3.165 and 3.162 respectively). ML-related studies those are indexed in ESCI show twice citation performance compared to DL. Conference papers in DL domain and journal papers in ML domain are superior in scientific interest to their counterparts with minor differences. Highest citation performance for ML studies is achieved for year 2014, while this peak is observed for 2017 for DL studies. We can conclude that both publication and citation rate for DL-related papers tend to increase and outperform ML-based studies in 5G domain by the means of citation metrics.
△ Less
Submitted 13 October, 2022;
originally announced October 2022.
-
Skip Letters for Short Supersequence of All Permutations
Authors:
Oliver Tan
Abstract:
A supersequence over a finite set is a sequence that contains as subsequence all permutations of the set. This paper defines an infinite array of methods to create supersequences of decreasing lengths. This yields the shortest known supersequences over larger sets. It also provides the best results asymptotically. It is based on a general proof using a new property called strong completeness. The…
▽ More
A supersequence over a finite set is a sequence that contains as subsequence all permutations of the set. This paper defines an infinite array of methods to create supersequences of decreasing lengths. This yields the shortest known supersequences over larger sets. It also provides the best results asymptotically. It is based on a general proof using a new property called strong completeness. The same technique also can be used to prove existing supersequences which combines the old and new ones into an unified conceptual framework.
△ Less
Submitted 17 January, 2022;
originally announced January 2022.
-
Linear Transmission of Composite Gaussian Measurements over a Fading Channel under Delay Constraints
Authors:
Onur Tan,
Deniz Gunduz,
Jesus Gomez Vilardebo
Abstract:
Delay constrained linear transmission (LT) strategies are considered for the transmission of composite Gaussian measurements over an additive white Gaussian noise fading channel under an average power constraint. If the channel state information (CSI) is known by both the encoder and decoder, the optimal LT scheme in terms of the average mean-square error distortion is characterized under a strict…
▽ More
Delay constrained linear transmission (LT) strategies are considered for the transmission of composite Gaussian measurements over an additive white Gaussian noise fading channel under an average power constraint. If the channel state information (CSI) is known by both the encoder and decoder, the optimal LT scheme in terms of the average mean-square error distortion is characterized under a strict delay constraint, and a graphical interpretation of the optimal power allocation strategy is presented. Then, for general delay constraints, two LT strategies are proposed based on the solution to a particular multiple measurements-parallel channels scenario. It is shown that the distortion decreases as the delay constraint is relaxed, and when the delay constraint is completely removed, both strategies achieve the optimal performance under certain matching conditions. If the CSI is known only by the decoder, the optimal LT strategy is derived under a strict delay constraint. The extension for general delay constraints is shown to be hard. As a first step towards understanding the structure of the optimal scheme in this case, it is shown that for the multiple measurements-parallel channels scenario, any LT scheme that uses only a one-to-one linear mapping between measurements and channels is suboptimal in general.
△ Less
Submitted 10 November, 2015; v1 submitted 26 May, 2015;
originally announced May 2015.
-
Increasing Smart Meter Privacy Through Energy Harvesting and Storage Devices
Authors:
Onur Tan,
Deniz Gunduz,
H. Vincent Poor
Abstract:
Smart meters are key elements for the operation of smart grids. By providing near realtime information on the energy consumption of individual users, smart meters increase the efficiency in generation, distribution and storage of energy in a smart grid. The ability of the utility provider to track users energy consumption inevitably leads to important threats to privacy. In this paper, privacy in…
▽ More
Smart meters are key elements for the operation of smart grids. By providing near realtime information on the energy consumption of individual users, smart meters increase the efficiency in generation, distribution and storage of energy in a smart grid. The ability of the utility provider to track users energy consumption inevitably leads to important threats to privacy. In this paper, privacy in a smart metering system is studied from an information theoretic perspective in the presence of energy harvesting and storage units. It is shown that energy harvesting provides increased privacy by diversifying the energy source, while a storage device can be used to increase both the energy efficiency and the privacy of the user. For given input load and energy harvesting rates, it is shown that there exists a trade-off between the information leakage rate, which is used to measure the privacy of the user, and the wasted energy rate, which is a measure of the energy-efficiency. The impact of the energy harvesting rate and the size of the storage device on this trade-off is also studied.
△ Less
Submitted 3 May, 2013;
originally announced May 2013.
