-
Dilated Strip Attention Network for Image Restoration
Authors:
Fangwei Hao,
Jiesheng Wu,
Ji Du,
Yinjie Wang,
Jing Xu
Abstract:
Image restoration is a long-standing task that seeks to recover the latent sharp image from its deteriorated counterpart. Due to the robust capacity of self-attention to capture long-range dependencies, transformer-based methods or some attention-based convolutional neural networks have demonstrated promising results on many image restoration tasks in recent years. However, existing attention modu…
▽ More
Image restoration is a long-standing task that seeks to recover the latent sharp image from its deteriorated counterpart. Due to the robust capacity of self-attention to capture long-range dependencies, transformer-based methods or some attention-based convolutional neural networks have demonstrated promising results on many image restoration tasks in recent years. However, existing attention modules encounters limited receptive fields or abundant parameters. In order to integrate contextual information more effectively and efficiently, in this paper, we propose a dilated strip attention network (DSAN) for image restoration. Specifically, to gather more contextual information for each pixel from its neighboring pixels in the same row or column, a dilated strip attention (DSA) mechanism is elaborately proposed. By employing the DSA operation horizontally and vertically, each location can harvest the contextual information from a much wider region. In addition, we utilize multi-scale receptive fields across different feature groups in DSA to improve representation learning. Extensive experiments show that our DSAN outperforms state-of-the-art algorithms on several image restoration tasks.
△ Less
Submitted 26 July, 2024;
originally announced July 2024.
-
GaitMA: Pose-guided Multi-modal Feature Fusion for Gait Recognition
Authors:
Fanxu Min,
Shaoxiang Guo,
Fan Hao,
Junyu Dong
Abstract:
Gait recognition is a biometric technology that recognizes the identity of humans through their walking patterns. Existing appearance-based methods utilize CNN or Transformer to extract spatial and temporal features from silhouettes, while model-based methods employ GCN to focus on the special topological structure of skeleton points. However, the quality of silhouettes is limited by complex occlu…
▽ More
Gait recognition is a biometric technology that recognizes the identity of humans through their walking patterns. Existing appearance-based methods utilize CNN or Transformer to extract spatial and temporal features from silhouettes, while model-based methods employ GCN to focus on the special topological structure of skeleton points. However, the quality of silhouettes is limited by complex occlusions, and skeletons lack dense semantic features of the human body. To tackle these problems, we propose a novel gait recognition framework, dubbed Gait Multi-model Aggregation Network (GaitMA), which effectively combines two modalities to obtain a more robust and comprehensive gait representation for recognition. First, skeletons are represented by joint/limb-based heatmaps, and features from silhouettes and skeletons are respectively extracted using two CNN-based feature extractors. Second, a co-attention alignment module is proposed to align the features by element-wise attention. Finally, we propose a mutual learning module, which achieves feature fusion through cross-attention, Wasserstein loss is further introduced to ensure the effective fusion of two modalities. Extensive experimental results demonstrate the superiority of our model on Gait3D, OU-MVLP, and CASIA-B.
△ Less
Submitted 20 July, 2024;
originally announced July 2024.
-
Large coordinate kernel attention network for lightweight image super-resolution
Authors:
Fangwei Hao,
Jiesheng Wu,
Haotian Lu,
Ji Du,
Jing Xu,
Xiaoxuan Xu
Abstract:
The multi-scale receptive field and large kernel attention (LKA) module have been shown to significantly improve performance in the lightweight image super-resolution task. However, existing lightweight super-resolution (SR) methods seldom pay attention to designing efficient building block with multi-scale receptive field for local modeling, and their LKA modules face a quadratic increase in comp…
▽ More
The multi-scale receptive field and large kernel attention (LKA) module have been shown to significantly improve performance in the lightweight image super-resolution task. However, existing lightweight super-resolution (SR) methods seldom pay attention to designing efficient building block with multi-scale receptive field for local modeling, and their LKA modules face a quadratic increase in computational and memory footprints as the convolutional kernel size increases. To address the first issue, we propose the multi-scale blueprint separable convolutions (MBSConv) as highly efficient building block with multi-scale receptive field, it can focus on the learning for the multi-scale information which is a vital component of discriminative representation. As for the second issue, we revisit the key properties of LKA in which we find that the adjacent direct interaction of local information and long-distance dependencies is crucial to provide remarkable performance. Thus, taking this into account and in order to mitigate the complexity of LKA, we propose a large coordinate kernel attention (LCKA) module which decomposes the 2D convolutional kernels of the depth-wise convolutional layers in LKA into horizontal and vertical 1-D kernels. LCKA enables the adjacent direct interaction of local information and long-distance dependencies not only in the horizontal direction but also in the vertical. Besides, LCKA allows for the direct use of extremely large kernels in the depth-wise convolutional layers to capture more contextual information, which helps to significantly improve the reconstruction performance, and it incurs lower computational complexity and memory footprints. Integrating MBSConv and LCKA, we propose a large coordinate kernel attention network (LCAN).
△ Less
Submitted 30 August, 2024; v1 submitted 15 May, 2024;
originally announced May 2024.
-
XPose: eXplainable Human Pose Estimation
Authors:
Luyu Qiu,
Jianing Li,
Lei Wen,
Chi Su,
Fei Hao,
Chen Jason Zhang,
Lei Chen
Abstract:
Current approaches in pose estimation primarily concentrate on enhancing model architectures, often overlooking the importance of comprehensively understanding the rationale behind model decisions. In this paper, we propose XPose, a novel framework that incorporates Explainable AI (XAI) principles into pose estimation. This integration aims to elucidate the individual contribution of each keypoint…
▽ More
Current approaches in pose estimation primarily concentrate on enhancing model architectures, often overlooking the importance of comprehensively understanding the rationale behind model decisions. In this paper, we propose XPose, a novel framework that incorporates Explainable AI (XAI) principles into pose estimation. This integration aims to elucidate the individual contribution of each keypoint to final prediction, thereby elevating the model's transparency and interpretability. Conventional XAI techniques have predominantly addressed tasks with single-target tasks like classification. Additionally, the application of Shapley value, a common measure in XAI, to pose estimation has been hindered by prohibitive computational demands.
To address these challenges, this work introduces an innovative concept called Group Shapley Value (GSV). This approach strategically organizes keypoints into clusters based on their interdependencies. Within these clusters, GSV meticulously calculates Shapley value for keypoints, while for inter-cluster keypoints, it opts for a more holistic group-level valuation. This dual-level computation framework meticulously assesses keypoint contributions to the final outcome, optimizing computational efficiency. Building on the insights into keypoint interactions, we devise a novel data augmentation technique known as Group-based Keypoint Removal (GKR). This method ingeniously removes individual keypoints during training phases, deliberately preserving those with strong mutual connections, thereby refining the model's predictive prowess for non-visible keypoints. The empirical validation of GKR across a spectrum of standard approaches attests to its efficacy. GKR's success demonstrates how using Explainable AI (XAI) can directly enhance pose estimation models.
△ Less
Submitted 18 March, 2024;
originally announced March 2024.
-
BoostER: Leveraging Large Language Models for Enhancing Entity Resolution
Authors:
Huahang Li,
Shuangyin Li,
Fei Hao,
Chen Jason Zhang,
Yuanfeng Song,
Lei Chen
Abstract:
Entity resolution, which involves identifying and merging records that refer to the same real-world entity, is a crucial task in areas like Web data integration. This importance is underscored by the presence of numerous duplicated and multi-version data resources on the Web. However, achieving high-quality entity resolution typically demands significant effort. The advent of Large Language Models…
▽ More
Entity resolution, which involves identifying and merging records that refer to the same real-world entity, is a crucial task in areas like Web data integration. This importance is underscored by the presence of numerous duplicated and multi-version data resources on the Web. However, achieving high-quality entity resolution typically demands significant effort. The advent of Large Language Models (LLMs) like GPT-4 has demonstrated advanced linguistic capabilities, which can be a new paradigm for this task. In this paper, we propose a demonstration system named BoostER that examines the possibility of leveraging LLMs in the entity resolution process, revealing advantages in both easy deployment and low cost. Our approach optimally selects a set of matching questions and poses them to LLMs for verification, then refines the distribution of entity resolution results with the response of LLMs. This offers promising prospects to achieve a high-quality entity resolution result for real-world applications, especially to individuals or small companies without the need for extensive model training or significant financial investment.
△ Less
Submitted 11 March, 2024;
originally announced March 2024.
-
Learning Discretized Bayesian Networks with GOMEA
Authors:
Damy M. F. Ha,
Tanja Alderliesten,
Peter A. N. Bosman
Abstract:
Bayesian networks model relationships between random variables under uncertainty and can be used to predict the likelihood of events and outcomes while incorporating observed evidence. From an eXplainable AI (XAI) perspective, such models are interesting as they tend to be compact. Moreover, captured relations can be directly inspected by domain experts. In practice, data is often real-valued. Unl…
▽ More
Bayesian networks model relationships between random variables under uncertainty and can be used to predict the likelihood of events and outcomes while incorporating observed evidence. From an eXplainable AI (XAI) perspective, such models are interesting as they tend to be compact. Moreover, captured relations can be directly inspected by domain experts. In practice, data is often real-valued. Unless assumptions of normality can be made, discretization is often required. The optimal discretization, however, depends on the relations modelled between the variables. This complicates learning Bayesian networks from data. For this reason, most literature focuses on learning conditional dependencies between sets of variables, called structure learning. In this work, we extend an existing state-of-the-art structure learning approach based on the Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA) to jointly learn variable discretizations. The proposed Discretized Bayesian Network GOMEA (DBN-GOMEA) obtains similar or better results than the current state-of-the-art when tasked to retrieve randomly generated ground-truth networks. Moreover, leveraging a key strength of evolutionary algorithms, we can straightforwardly perform DBN learning multi-objectively. We show how this enables incorporating expert knowledge in a uniquely insightful fashion, finding multiple DBNs that trade-off complexity, accuracy, and the difference with a pre-determined expert network.
△ Less
Submitted 19 February, 2024;
originally announced February 2024.
-
On Leveraging Large Language Models for Enhancing Entity Resolution
Authors:
Huahang Li,
Longyu Feng,
Shuangyin Li,
Fei Hao,
Chen Jason Zhang,
Yuanfeng Song,
Lei Chen
Abstract:
Entity resolution, the task of identifying and consolidating records that pertain to the same real-world entity, plays a pivotal role in various sectors such as e-commerce, healthcare, and law enforcement. The emergence of Large Language Models (LLMs) like GPT-4 has introduced a new dimension to this task, leveraging their advanced linguistic capabilities. This paper explores the potential of LLMs…
▽ More
Entity resolution, the task of identifying and consolidating records that pertain to the same real-world entity, plays a pivotal role in various sectors such as e-commerce, healthcare, and law enforcement. The emergence of Large Language Models (LLMs) like GPT-4 has introduced a new dimension to this task, leveraging their advanced linguistic capabilities. This paper explores the potential of LLMs in the entity resolution process, shedding light on both their advantages and the computational complexities associated with large-scale matching. We introduce strategies for the efficient utilization of LLMs, including the selection of an optimal set of matching questions, namely MQsSP, which is proved to be a NP-hard problem. Our approach optimally chooses the most effective matching questions while keep consumption limited to your budget . Additionally, we propose a method to adjust the distribution of possible partitions after receiving responses from LLMs, with the goal of reducing the uncertainty of entity resolution. We evaluate the effectiveness of our approach using entropy as a metric, and our experimental results demonstrate the efficiency and effectiveness of our proposed methods, offering promising prospects for real-world applications.
△ Less
Submitted 7 January, 2024;
originally announced January 2024.
-
Spoofing Against Spoofing: Towards Caller ID Verification In Heterogeneous Telecommunication Systems
Authors:
Shen Wang,
Mahshid Delavar,
Muhammad Ajmal Azad,
Farshad Nabizadeh,
Steve Smith,
Feng Hao
Abstract:
Caller ID spoofing is a global industry problem and often acts as a critical enabler for telephone fraud. To address this problem, the Federal Communications Commission (FCC) has mandated telecom providers in the US to implement STIR/SHAKEN, an industry-driven solution based on digital signatures. STIR/SHAKEN relies on a public key infrastructure (PKI) to manage digital certificates, but scaling u…
▽ More
Caller ID spoofing is a global industry problem and often acts as a critical enabler for telephone fraud. To address this problem, the Federal Communications Commission (FCC) has mandated telecom providers in the US to implement STIR/SHAKEN, an industry-driven solution based on digital signatures. STIR/SHAKEN relies on a public key infrastructure (PKI) to manage digital certificates, but scaling up this PKI for the global telecom industry is extremely difficult, if not impossible. Furthermore, it only works with IP-based systems (e.g., SIP), leaving the traditional non-IP systems (e.g., SS7) unprotected. So far the alternatives to the STIR/SHAKEN have not been sufficiently studied. In this paper, we propose a PKI-free solution, called Caller ID Verification (CIV). CIV authenticates the caller ID based on a challenge-response process instead of digital signatures, hence requiring no PKI. It supports both IP and non-IP systems. Perhaps counter-intuitively, we show that number spoofing can be leveraged, in conjunction with Dual-Tone Multi-Frequency (DTMF), to efficiently implement the challenge-response process, i.e., using spoofing to fight against spoofing. We implement CIV for VoIP, cellular, and landline phones across heterogeneous networks (SS7/SIP) by only updating the software on the user's phone. This is the first caller ID authentication solution with working prototypes for all three types of telephone systems in the current telecom architecture. Finally, we show how the implementation of CIV can be optimized by integrating it into telecom clouds as a service, which users may subscribe to.
△ Less
Submitted 24 September, 2023; v1 submitted 9 June, 2023;
originally announced June 2023.
-
Human-imperceptible, Machine-recognizable Images
Authors:
Fusheng Hao,
Fengxiang He,
Yikai Wang,
Fuxiang Wu,
Jing Zhang,
Jun Cheng,
Dacheng Tao
Abstract:
Massive human-related data is collected to train neural networks for computer vision tasks. A major conflict is exposed relating to software engineers between better developing AI systems and distancing from the sensitive training data. To reconcile this conflict, this paper proposes an efficient privacy-preserving learning paradigm, where images are first encrypted to become ``human-imperceptible…
▽ More
Massive human-related data is collected to train neural networks for computer vision tasks. A major conflict is exposed relating to software engineers between better developing AI systems and distancing from the sensitive training data. To reconcile this conflict, this paper proposes an efficient privacy-preserving learning paradigm, where images are first encrypted to become ``human-imperceptible, machine-recognizable'' via one of the two encryption strategies: (1) random shuffling to a set of equally-sized patches and (2) mixing-up sub-patches of the images. Then, minimal adaptations are made to vision transformer to enable it to learn on the encrypted images for vision tasks, including image classification and object detection. Extensive experiments on ImageNet and COCO show that the proposed paradigm achieves comparable accuracy with the competitive methods. Decrypting the encrypted images requires solving an NP-hard jigsaw puzzle or an ill-posed inverse problem, which is empirically shown intractable to be recovered by various attackers, including the powerful vision transformer-based attacker. We thus show that the proposed paradigm can ensure the encrypted images have become human-imperceptible while preserving machine-recognizable information. The code is available at \url{https://github.com/FushengHao/PrivacyPreservingML.}
△ Less
Submitted 6 June, 2023;
originally announced June 2023.
-
Spying on the Spy: Security Analysis of Hidden Cameras
Authors:
Samuel Herodotou,
Feng Hao
Abstract:
Hidden cameras, also called spy cameras, are surveillance tools commonly used to spy on people without their knowledge. Whilst previous studies largely focused on investigating the detection of such a camera and the privacy implications, the security of the camera itself has received limited attention. Compared with ordinary IP cameras, spy cameras are normally sold in bulk at cheap prices and are…
▽ More
Hidden cameras, also called spy cameras, are surveillance tools commonly used to spy on people without their knowledge. Whilst previous studies largely focused on investigating the detection of such a camera and the privacy implications, the security of the camera itself has received limited attention. Compared with ordinary IP cameras, spy cameras are normally sold in bulk at cheap prices and are ubiquitously deployed in hidden places within homes and workplaces. A security compromise of these cameras can have severe consequences. In this paper, we analyse a generic IP camera module, which has been packaged and re-branded for sale by several spy camera vendors. The module is controlled by mobile phone apps. By analysing the Android app and the traffic data, we reverse-engineered the security design of the whole system, including the module's Linux OS environment, the file structure, the authentication mechanism, the session management, and the communication with a remote server. Serious vulnerabilities have been identified in every component. Combined together, they allow an adversary to take complete control of a spy camera from anywhere over the Internet, enabling arbitrary code execution. This is possible even if the camera is behind a firewall. All that an adversary needs to launch an attack is the camera's serial number, which users sometimes unknowingly share in online reviews. We responsibly disclosed our findings to the manufacturer. Whilst the manufacturer acknowledged our work, they showed no intention to fix the problems. Patching or recalling the affected cameras is infeasible due to complexities in the supply chain. However, it is prudent to assume that bad actors have already been exploiting these flaws. We provide details of the identified vulnerabilities in order to raise public awareness, especially on the grave danger of disclosing a spy camera's serial number.
△ Less
Submitted 1 June, 2023;
originally announced June 2023.
-
Can You See Me Now? A Measurement Study of Zoom, Webex, and Meet
Authors:
Hyunseok Chang,
Matteo Varvello,
Fang Hao,
Sarit Mukherjee
Abstract:
Since the outbreak of the COVID-19 pandemic, videoconferencing has become the default mode of communication in our daily lives at homes, workplaces and schools, and it is likely to remain an important part of our lives in the post-pandemic world. Despite its significance, there has not been any systematic study characterizing the user-perceived performance of existing videoconferencing systems oth…
▽ More
Since the outbreak of the COVID-19 pandemic, videoconferencing has become the default mode of communication in our daily lives at homes, workplaces and schools, and it is likely to remain an important part of our lives in the post-pandemic world. Despite its significance, there has not been any systematic study characterizing the user-perceived performance of existing videoconferencing systems other than anecdotal reports. In this paper, we present a detailed measurement study that compares three major videoconferencing systems: Zoom, Webex and Google Meet. Our study is based on 48 hours' worth of more than 700 videoconferencing sessions, which were created with a mix of emulated videoconferencing clients deployed in the cloud, as well as real mobile devices running from a residential network. We find that the existing videoconferencing systems vary in terms of geographic scope, which in turns determines streaming lag experienced by users. We also observe that streaming rate can change under different conditions (e.g., number of users in a session, mobile device status, etc), which affects user-perceived streaming quality. Beyond these findings, our measurement methodology can enable reproducible benchmark analysis for any types of comparative or longitudinal study on available videoconferencing systems.
△ Less
Submitted 27 September, 2021;
originally announced September 2021.
-
HMSG: Heterogeneous Graph Neural Network based on Metapath Subgraph Learning
Authors:
Xinjun Cai,
Jiaxing Shang,
Fei Hao,
Dajiang Liu,
Linjiang Zheng
Abstract:
Many real-world data can be represented as heterogeneous graphs with different types of nodes and connections. Heterogeneous graph neural network model aims to embed nodes or subgraphs into low-dimensional vector space for various downstream tasks such as node classification, link prediction, etc. Although several models were proposed recently, they either only aggregate information from the same…
▽ More
Many real-world data can be represented as heterogeneous graphs with different types of nodes and connections. Heterogeneous graph neural network model aims to embed nodes or subgraphs into low-dimensional vector space for various downstream tasks such as node classification, link prediction, etc. Although several models were proposed recently, they either only aggregate information from the same type of neighbors, or just indiscriminately treat homogeneous and heterogeneous neighbors in the same way. Based on these observations, we propose a new heterogeneous graph neural network model named HMSG to comprehensively capture structural, semantic and attribute information from both homogeneous and heterogeneous neighbors. Specifically, we first decompose the heterogeneous graph into multiple metapath-based homogeneous and heterogeneous subgraphs, and each subgraph associates specific semantic and structural information. Then message aggregation methods are applied to each subgraph independently, so that information can be learned in a more targeted and efficient manner. Through a type-specific attribute transformation, node attributes can also be transferred among different types of nodes. Finally, we fuse information from subgraphs together to get the complete representation. Extensive experiments on several datasets for node classification, node clustering and link prediction tasks show that HMSG achieves the best performance in all evaluation metrics than state-of-the-art baselines.
△ Less
Submitted 7 September, 2021;
originally announced September 2021.
-
Formal Modelling and Security Analysis of Bitcoin's Payment Protocol
Authors:
Paolo Modesti,
Siamak F. Shahandashti,
Patrick McCorry,
Feng Hao
Abstract:
The Payment Protocol standard BIP70, specifying how payments in Bitcoin are performed by merchants and customers, is supported by the largest payment processors and most widely-used wallets. The protocol has been shown to be vulnerable to refund attacks due to lack of authentication of the refund addresses. In this paper, we give the first formal model of the protocol and formalise the refund addr…
▽ More
The Payment Protocol standard BIP70, specifying how payments in Bitcoin are performed by merchants and customers, is supported by the largest payment processors and most widely-used wallets. The protocol has been shown to be vulnerable to refund attacks due to lack of authentication of the refund addresses. In this paper, we give the first formal model of the protocol and formalise the refund address security goals for the protocol, namely refund address authentication and secrecy. The formal model utilises communication channels as abstractions conveying security goals on which the protocol modeller and verifier can rely. We analyse the Payment Protocol confirming that it is vulnerable to an attack violating the refund address authentication security goal. Moreover, we present a concrete protocol revision proposal supporting the merchant with publicly verifiable evidence that can mitigate the attack. We verify that the revised protocol meets the security goals defined for the refund address. Hence, we demonstrate that the revised protocol is secure, not only against the existing attacks, but also against any further attacks violating the formalised security goals.
△ Less
Submitted 15 March, 2021;
originally announced March 2021.
-
Anti-Counterfeiting for Polymer Banknotes Based on Polymer Substrate Fingerprinting
Authors:
Shen Wang,
Ehsan Toreini,
Feng Hao
Abstract:
Polymer banknotes are the trend for printed currency and have been adopted by more than fifty countries worldwide. However, over the past years, the quantity of polymer counterfeits has been increasing, so has the quality of counterfeits. This shows that the initial advantage of bringing a new polymer technology to fight against counterfeiting is reducing. To maintain one step ahead of counterfeit…
▽ More
Polymer banknotes are the trend for printed currency and have been adopted by more than fifty countries worldwide. However, over the past years, the quantity of polymer counterfeits has been increasing, so has the quality of counterfeits. This shows that the initial advantage of bringing a new polymer technology to fight against counterfeiting is reducing. To maintain one step ahead of counterfeiters, we propose a novel anti-counterfeiting technique called Polymer Substrate Fingerprinting (PSF). Our technique is built based on the observation that the opacity coating, a critical step during the production of polymer notes, is a stochastic manufacturing process, leaving uneven thickness in the coating layer and the random dispersion of impurities from the ink. The imperfections in the coating layer result in random translucent patterns when a polymer banknote is back-lit by a light source. We show these patterns can be reliably captured by a commodity negative-film scanner and processed into a compact fingerprint to uniquely identify each banknote. Using an extensive dataset of 6,200 sample images collected from 340 UK banknotes, we show that our method can reliably authenticate banknotes, and is robust against rough daily handling of banknotes. Furthermore, we show the extracted fingerprints contain around 900 bits of entropy, which makes it extremely scalable to identify every polymer note circulated globally. As compared with previous or existing anti-counterfeiting mechanisms for banknotes, our method has a distinctive advantage: it ensures that even in the extreme case when counterfeiters have procured the same printing equipment and ink as used by a legitimate government, counterfeiting banknotes remains infeasible because of the difficulty to replicate a stochastic manufacturing process.
△ Less
Submitted 23 March, 2021; v1 submitted 10 March, 2021;
originally announced March 2021.
-
MSA-MIL: A deep residual multiple instance learning model based on multi-scale annotation for classification and visualization of glomerular spikes
Authors:
Yilin Chen,
Ming Li,
Yongfei Wu,
Xueyu Liu,
Fang Hao,
Daoxiang Zhou,
Xiaoshuang Zhou,
Chen Wang
Abstract:
Membranous nephropathy (MN) is a frequent type of adult nephrotic syndrome, which has a high clinical incidence and can cause various complications. In the biopsy microscope slide of membranous nephropathy, spikelike projections on the glomerular basement membrane is a prominent feature of the MN. However, due to the whole biopsy slide contains large number of glomeruli, and each glomerulus includ…
▽ More
Membranous nephropathy (MN) is a frequent type of adult nephrotic syndrome, which has a high clinical incidence and can cause various complications. In the biopsy microscope slide of membranous nephropathy, spikelike projections on the glomerular basement membrane is a prominent feature of the MN. However, due to the whole biopsy slide contains large number of glomeruli, and each glomerulus includes many spike lesions, the pathological feature of the spikes is not obvious. It thus is time-consuming for doctors to diagnose glomerulus one by one and is difficult for pathologists with less experience to diagnose. In this paper, we establish a visualized classification model based on the multi-scale annotation multi-instance learning (MSA-MIL) to achieve glomerular classification and spikes visualization. The MSA-MIL model mainly involves three parts. Firstly, U-Net is used to extract the region of the glomeruli to ensure that the features learned by the succeeding algorithm are focused inside the glomeruli itself. Secondly, we use MIL to train an instance-level classifier combined with MSA method to enhance the learning ability of the network by adding a location-level labeled reinforced dataset, thereby obtaining an example-level feature representation with rich semantics. Lastly, the predicted scores of each tile in the image are summarized to obtain glomerular classification and visualization of the classification results of the spikes via the usage of sliding window method. The experimental results confirm that the proposed MSA-MIL model can effectively and accurately classify normal glomeruli and spiked glomerulus and visualize the position of spikes in the glomerulus. Therefore, the proposed model can provide a good foundation for assisting the clinical doctors to diagnose the glomerular membranous nephropathy.
△ Less
Submitted 18 July, 2020; v1 submitted 1 July, 2020;
originally announced July 2020.
-
Game Theoretical Approach to Sequential Hypothesis Test with Byzantine Sensors
Authors:
Zishuo Li,
Yilin Mo,
Fei Hao
Abstract:
In this paper, we consider the problem of sequential binary hypothesis test in adversary environment based on observations from s sensors, with the caveat that a subset of c sensors is compromised by an adversary, whose observations can be manipulated arbitrarily. We choose the asymptotic Average Sample Number (ASN) required to reach a certain level of error probability as the performance metric o…
▽ More
In this paper, we consider the problem of sequential binary hypothesis test in adversary environment based on observations from s sensors, with the caveat that a subset of c sensors is compromised by an adversary, whose observations can be manipulated arbitrarily. We choose the asymptotic Average Sample Number (ASN) required to reach a certain level of error probability as the performance metric of the system. The problem is cast as a game between the detector and the adversary, where the detector aims to optimize the system performance while the adversary tries to deteriorate it. We propose a pair of flip attack strategy and voting hypothesis testing rule and prove that they form an equilibrium strategy pair for the game. We further investigate the performance of our proposed detection scheme with unknown number of compromised sensors and corroborate our result with simulation.
△ Less
Submitted 6 September, 2019;
originally announced September 2019.
-
DOMtegrity: Ensuring Web Page Integrity against Malicious Browser Extensions
Authors:
Ehsan Toreini,
Maryam Mehrnezhad,
Siamak F. Shahandashti,
Feng Hao
Abstract:
In this paper, we address an unsolved problem in the real world: how to ensure the integrity of the web content in a browser in the presence of malicious browser extensions? The problem of exposing confidential user credentials to malicious extensions has been widely understood, which has prompted major banks to deploy two-factor authentication. However, the importance of the `integrity' of the we…
▽ More
In this paper, we address an unsolved problem in the real world: how to ensure the integrity of the web content in a browser in the presence of malicious browser extensions? The problem of exposing confidential user credentials to malicious extensions has been widely understood, which has prompted major banks to deploy two-factor authentication. However, the importance of the `integrity' of the web content has received little attention. We implement two attacks on real-world online banking websites and show that ignoring the `integrity' of the web content can fundamentally defeat two-factor solutions. To address this problem, we propose a cryptographic protocol called DOMtegrity to ensure the end-to-end integrity of the DOM structure of a web page from delivering at a web server to the rendering of the page in the user's browser. DOMtegrity is the first solution that protects DOM integrity without modifying the browser architecture or requiring extra hardware. It works by exploiting subtle yet important differences between browser extensions and in-line JavaScript code. We show how DOMtegrity prevents the earlier attacks and a whole range of man-in-the-browser (MITB) attacks. We conduct extensive experiments on more than 14,000 real-world extensions to evaluate the effectiveness of DOMtegrity.
△ Less
Submitted 30 May, 2019;
originally announced May 2019.
-
On legitimate mining of cryptocurrency in the browser - a feasibility study
Authors:
Saulius Venskutonis,
Feng Hao,
Matthew Collison
Abstract:
Cryptocurrency mining in the browser has the potential to provide a new pay-as-you-go monetisation mechanism for consuming digital media over the Web. However, browser mining has recently received strong criticism due to illegitimate use of mining scripts in several popular websites (a practice called cryptojacking). Here we provide the first feasibility study of browser mining as a legitimate mea…
▽ More
Cryptocurrency mining in the browser has the potential to provide a new pay-as-you-go monetisation mechanism for consuming digital media over the Web. However, browser mining has recently received strong criticism due to illegitimate use of mining scripts in several popular websites (a practice called cryptojacking). Here we provide the first feasibility study of browser mining as a legitimate means of monetisation in terms of revenue, user consent and user experience within a specially built website. Our results compare browser mining to display advertisement and indicate browser mining provides a preferable user experience to advertising when the hash rate is user-adjustable. Furthermore, over 60% of participants would select browser mining over advertisement if they were invested in the ecosystem by obtaining half of the mined cryptocurrency. Our estimations show that browser mining currently generates revenue at a rate 46 times less than advertisement, however we would expect that gap to decrease as we observed a significant drop in mining difficulty after our tested cryptocurrency implemented ASIC-resistant mining measures. Overall, based on our results we find browser mining to be a legitimate alternative to display advertisement and conclude by discussing its current limitations and potential applications.
△ Less
Submitted 4 January, 2019; v1 submitted 10 December, 2018;
originally announced December 2018.
-
Anchor-based Nearest Class Mean Loss for Convolutional Neural Networks
Authors:
Fusheng Hao,
Jun Cheng,
Lei Wang,
Xinchao Wang,
Jianzhong Cao,
Xiping Hu,
Dapeng Tao
Abstract:
Discriminative features are critical for machine learning applications. Most existing deep learning approaches, however, rely on convolutional neural networks (CNNs) for learning features, whose discriminant power is not explicitly enforced. In this paper, we propose a novel approach to train deep CNNs by imposing the intra-class compactness and the inter-class separability, so as to enhance the l…
▽ More
Discriminative features are critical for machine learning applications. Most existing deep learning approaches, however, rely on convolutional neural networks (CNNs) for learning features, whose discriminant power is not explicitly enforced. In this paper, we propose a novel approach to train deep CNNs by imposing the intra-class compactness and the inter-class separability, so as to enhance the learned features' discriminant power. To this end, we introduce anchors, which are predefined vectors regarded as the centers for each class and fixed during training. Discriminative features are obtained by constraining the deep CNNs to map training samples to the corresponding anchors as close as possible. We propose two principles to select the anchors, and measure the proximity of two points using the Euclidean and cosine distance metric functions, which results in two novel loss functions. These loss functions require no sample pairs or triplets and can be efficiently optimized by batch stochastic gradient descent. We test the proposed method on three benchmark image classification datasets and demonstrate its promising results.
△ Less
Submitted 22 April, 2018;
originally announced April 2018.
-
Cryptanalysis of a Chaotic Image Encryption Algorithm Based on Information Entropy
Authors:
Chengqing Li,
Dongdong Lin,
Bingbing Feng,
Jinhu Lü,
Feng Hao
Abstract:
Recently, a chaotic image encryption algorithm based on information entropy (IEAIE) was proposed. This paper scrutinizes the security properties of the algorithm and evaluates the validity of the used quantifiable security metrics. When the round number is only one, the equivalent secret key of every basic operation of IEAIE can be recovered with a differential attack separately. Some common insec…
▽ More
Recently, a chaotic image encryption algorithm based on information entropy (IEAIE) was proposed. This paper scrutinizes the security properties of the algorithm and evaluates the validity of the used quantifiable security metrics. When the round number is only one, the equivalent secret key of every basic operation of IEAIE can be recovered with a differential attack separately. Some common insecurity problems in the field of chaotic image encryption are found in IEAIE, e.g. the short orbits of the digital chaotic system and the invalid sensitivity mechanism built on information entropy of the plain image. Even worse, each security metric is questionable, which undermines the security credibility of IEAIE. Hence, IEAIE can only serve as a counterexample for illustrating common pitfalls in designing secure communication method for image data.
△ Less
Submitted 25 November, 2018; v1 submitted 27 March, 2018;
originally announced March 2018.
-
Analysing and Patching SPEKE in ISO/IEC
Authors:
Feng Hao,
Roberto Metere,
Siamak F. Shahandashti,
Changyu Dong
Abstract:
Simple Password Exponential Key Exchange (SPEKE) is a well-known Password Authenticated Key Exchange (PAKE) protocol that has been used in Blackberry phones for secure messaging and Entrust's TruePass end-to-end web products. It has also been included into international standards such as ISO/IEC 11770-4 and IEEE P1363.2. In this paper, we analyse the SPEKE protocol as specified in the ISO/IEC and…
▽ More
Simple Password Exponential Key Exchange (SPEKE) is a well-known Password Authenticated Key Exchange (PAKE) protocol that has been used in Blackberry phones for secure messaging and Entrust's TruePass end-to-end web products. It has also been included into international standards such as ISO/IEC 11770-4 and IEEE P1363.2. In this paper, we analyse the SPEKE protocol as specified in the ISO/IEC and IEEE standards. We identify that the protocol is vulnerable to two new attacks: an impersonation attack that allows an attacker to impersonate a user without knowing the password by launching two parallel sessions with the victim, and a key-malleability attack that allows a man-in-the-middle (MITM) to manipulate the session key without being detected by the end users. Both attacks have been acknowledged by the technical committee of ISO/IEC SC 27, and ISO/IEC 11770-4 revised as a result. We propose a patched SPEKE called P-SPEKE and present a formal analysis in the Applied Pi Calculus using ProVerif to show that the proposed patch prevents both attacks. The proposed patch has been included into the latest revision of ISO/IEC 11770-4 published in 2017.
△ Less
Submitted 4 April, 2018; v1 submitted 13 February, 2018;
originally announced February 2018.
-
Cryptanalyzing an image encryption algorithm based on autoblocking and electrocardiography
Authors:
Chengqing Li,
Dongdong Lin,
Jinhu Lü,
Feng Hao
Abstract:
This paper analyzes the security of an image encryption algorithm proposed by Ye and Huang [\textit{IEEE MultiMedia}, vol. 23, pp. 64-71, 2016]. The Ye-Huang algorithm uses electrocardiography (ECG) signals to generate the initial key for a chaotic system and applies an autoblocking method to divide a plain image into blocks of certain sizes suitable for subsequent encryption. The designers claime…
▽ More
This paper analyzes the security of an image encryption algorithm proposed by Ye and Huang [\textit{IEEE MultiMedia}, vol. 23, pp. 64-71, 2016]. The Ye-Huang algorithm uses electrocardiography (ECG) signals to generate the initial key for a chaotic system and applies an autoblocking method to divide a plain image into blocks of certain sizes suitable for subsequent encryption. The designers claimed that the proposed algorithm is "strong and flexible enough for practical applications". In this paper, we perform a thorough analysis of their algorithm from the view point of modern cryptography. We find it is vulnerable to the known plaintext attack: based on one pair of a known plain-image and its corresponding cipher-image, an adversary is able to derive a mask image, which can be used as an equivalent secret key to successfully decrypt other cipher-images encrypted under the same key with a non-negligible probability of 1/256. Using this as a typical counterexample, we summarize security defects in the design of the Ye-Huang algorithm. The lessons are generally applicable to many other image encryption schemes.
△ Less
Submitted 8 June, 2018; v1 submitted 6 November, 2017;
originally announced November 2017.
-
Botnet in the Browser: Understanding Threats Caused by Malicious Browser Extensions
Authors:
Raffaello Perrotta,
Feng Hao
Abstract:
Browser extensions have been established as a common feature present in modern browsers. However, some extension systems risk exposing APIs which are too permissive and cohesive with the browser's internal structure, thus leaving a hole for malicious developers to exploit security critical functionality within the browser itself. In this paper, we raise the awareness of the threats caused by brows…
▽ More
Browser extensions have been established as a common feature present in modern browsers. However, some extension systems risk exposing APIs which are too permissive and cohesive with the browser's internal structure, thus leaving a hole for malicious developers to exploit security critical functionality within the browser itself. In this paper, we raise the awareness of the threats caused by browser extensions by presenting a botnet framework based on malicious extensions installed in the user's browser, and an exhaustive range of attacks that can be launched in this framework. We systematically categorize, describe and implement these attacks against Chrome, Firefox and Firefox-for-Android, and verify experiments on Windows, Linux and Android systems. To the best of our knowledge, this paper presents to date the most comprehensive analysis about the threats of botnet in modern browsers due to the over-privileged capabilities possessed by browser extensions. We also discuss countermeasures to the identified problems.
△ Less
Submitted 27 September, 2017;
originally announced September 2017.
-
An Algorithm of Parking Planning for Smart Parking System
Authors:
Xuejian Zhao,
Kui Zhao,
Feng Ha
Abstract:
There are so many vehicles in the world and the number of vehicles is increasing rapidly. To alleviate the parking problems caused by that, the smart parking system has been developed. The parking planning is one of the most important parts of it. An effective parking planning strategy makes the better use of parking resources possible. In this paper, we present a feasible method to do parking pla…
▽ More
There are so many vehicles in the world and the number of vehicles is increasing rapidly. To alleviate the parking problems caused by that, the smart parking system has been developed. The parking planning is one of the most important parts of it. An effective parking planning strategy makes the better use of parking resources possible. In this paper, we present a feasible method to do parking planning. We transform the parking planning problem into a kind of linear assignment problem. We take vehicles as jobs and parking spaces as agents. We take distances between vehicles and parking spaces as costs for agents doing jobs. Then we design an algorithm for this particular assignment problem and solve the parking planning problem. The method proposed can give timely and efficient guide information to vehicles for a real time smart parking system. Finally, we show the effectiveness of the method with experiments over some data, which can simulate the situation of doing parking planning in the real world.
△ Less
Submitted 26 August, 2017;
originally announced August 2017.
-
Texture to the Rescue: Practical Paper Fingerprinting based on Texture Patterns
Authors:
Ehsan Toreini,
Siamak F. Shahandashti,
Feng Hao
Abstract:
In this paper, we propose a novel paper fingerprinting technique based on analyzing the translucent patterns revealed when a light source shines through the paper. These patterns represent the inherent texture of paper, formed by the random interleaving of wooden particles during the manufacturing process. We show these patterns can be easily captured by a commodity camera and condensed into to a…
▽ More
In this paper, we propose a novel paper fingerprinting technique based on analyzing the translucent patterns revealed when a light source shines through the paper. These patterns represent the inherent texture of paper, formed by the random interleaving of wooden particles during the manufacturing process. We show these patterns can be easily captured by a commodity camera and condensed into to a compact 2048-bit fingerprint code. Prominent works in this area (Nature 2005, IEEE S&P 2009, CCS 2011) have all focused on fingerprinting paper based on the paper "surface". We are motivated by the observation that capturing the surface alone misses important distinctive features such as the non-even thickness, the random distribution of impurities, and different materials in the paper with varying opacities. Through experiments, we demonstrate that the embedded paper texture provides a more reliable source for fingerprinting than features on the surface. Based on the collected datasets, we achieve 0% false rejection and 0% false acceptance rates. We further report that our extracted fingerprints contain 807 degrees-of-freedom (DoF), which is much higher than the 249 DoF with iris codes (that have the same size of 2048 bits). The high amount of DoF for texture-based fingerprints makes our method extremely scalable for recognition among very large databases; it also allows secure usage of the extracted fingerprint in privacy-preserving authentication schemes based on error correction techniques.
△ Less
Submitted 7 May, 2019; v1 submitted 6 May, 2017;
originally announced May 2017.
-
Stealing PINs via Mobile Sensors: Actual Risk versus User Perception
Authors:
Maryam Mehrnezhad,
Ehsan Toreini,
Siamak F. Shahandashti,
Feng Hao
Abstract:
In this paper, we present the actual risks of stealing user PINs by using mobile sensors versus the perceived risks by users. First, we propose PINlogger.js which is a JavaScript-based side channel attack revealing user PINs on an Android mobile phone. In this attack, once the user visits a website controlled by an attacker, the JavaScript code embedded in the web page starts listening to the moti…
▽ More
In this paper, we present the actual risks of stealing user PINs by using mobile sensors versus the perceived risks by users. First, we propose PINlogger.js which is a JavaScript-based side channel attack revealing user PINs on an Android mobile phone. In this attack, once the user visits a website controlled by an attacker, the JavaScript code embedded in the web page starts listening to the motion and orientation sensor streams without needing any permission from the user. By analysing these streams, it infers the user's PIN using an artificial neural network. Based on a test set of fifty 4-digit PINs, PINlogger.js is able to correctly identify PINs in the first attempt with a success rate of 74% which increases to 86 and 94% in the second and third attempts, respectively. The high success rates of stealing user PINs on mobile devices via JavaScript indicate a serious threat to user security. With the technical understanding of the information leakage caused by mobile phone sensors, we then study users' perception of the risks associated with these sensors. We design user studies to measure the general familiarity with different sensors and their functionality, and to investigate how concerned users are about their PIN being discovered by an app that has access to all these sensors. Our studies show that there is significant disparity between the actual and perceived levels of threat with regard to the compromise of the user PIN. We confirm our results by interviewing our participants using two different approaches, within-subject and between-subject, and compare the results. We discuss how this observation, along with other factors, renders many academic and industry solutions ineffective in preventing such side channel attacks.
△ Less
Submitted 18 April, 2017; v1 submitted 18 May, 2016;
originally announced May 2016.
-
TouchSignatures: Identification of User Touch Actions and PINs Based on Mobile Sensor Data via JavaScript
Authors:
Maryam Mehrnezhad,
Ehsan Toreini,
Siamak F. Shahandashti,
Feng Hao
Abstract:
Conforming to W3C specifications, mobile web browsers allow JavaScript code in a web page to access motion and orientation sensor data without the user's permission. The associated risks to user security and privacy are however not considered in W3C specifications. In this work, for the first time, we show how user security can be compromised using these sensor data via browser, despite that the d…
▽ More
Conforming to W3C specifications, mobile web browsers allow JavaScript code in a web page to access motion and orientation sensor data without the user's permission. The associated risks to user security and privacy are however not considered in W3C specifications. In this work, for the first time, we show how user security can be compromised using these sensor data via browser, despite that the data rate is 3 to 5 times slower than what is available in app. We examine multiple popular browsers on Android and iOS platforms and study their policies in granting permissions to JavaScript code with respect to access to motion and orientation sensor data. Based on our observations, we identify multiple vulnerabilities, and propose TouchSignatures which implements an attack where malicious JavaScript code on an attack tab listens to such sensor data measurements. Based on these streams, TouchSignatures is able to distinguish the user's touch actions (i.e., tap, scroll, hold, and zoom) and her PINs, allowing a remote website to learn the client-side user activities. We demonstrate the practicality of this attack by collecting data from real users and reporting high success rates using our proof-of-concept implementations. We also present a set of potential solutions to address the vulnerabilities. The W3C community and major mobile browser vendors including Mozilla, Google, Apple and Opera have acknowledge our work and are implementing some of our proposed countermeasures.
△ Less
Submitted 12 February, 2016;
originally announced February 2016.
