pyDRMetrics - A Python toolkit for dimensionality reduction quality assessment

Yinsheng Zhang; Qian Shang; Guoming Zhang

doi:10.1016/j.heliyon.2021.e06199

pyDRMetrics - A Python toolkit for dimensionality reduction quality assessment

Heliyon. 2021 Feb 6;7(2):e06199. doi: 10.1016/j.heliyon.2021.e06199. eCollection 2021 Feb.

Authors

Yinsheng Zhang^{1

2}, Qian Shang^{3

2}, Guoming Zhang^{4

5}

Affiliations

¹ School of Management and E-Business, Zhejiang Gongshang University, Hangzhou 310018, China.
² School of Information Sciences, University of Illinois at Urbana Champaign, Champaign, IL 61820-6211, USA.
³ School of Management, Hangzhou Dianzi University, Hangzhou 310018, China.
⁴ Pediatric Retinal Surgery Department, Shenzhen Eye Hospital, Shenzhen 518040, China.
⁵ Shenzhen Key Ophthalmic Laboratory, The Second Affiliated Hospital of Jinan University, Shenzhen 518040, China.

Abstract

High-dimensional data are pervasive in this bigdata era. To avoid the curse of the dimensionality problem, various dimensionality reduction (DR) algorithms have been proposed. To facilitate systematic DR quality comparison and assessment, this paper reviews related metrics and develops an open-source Python package pyDRMetrics. Supported metrics include reconstruction error, distance matrix, residual variance, ranking matrix, co-ranking matrix, trustworthiness, continuity, co-k-nearest neighbor size, LCMC (local continuity meta criterion), and rank-based local/global properties. pyDRMetrics provides a native Python class and a web-oriented API. A case study of mass spectra is conducted to demonstrate the package functions. A web GUI wrapper is also published to support user-friendly B/S applications.

Keywords: Co-k-nearest neighbor; Co-ranking matrix; Dimensionality reduction; Distance matrix; Reconstruction error.