Functional form estimation using oblique projection matrices for LS-SVM regression models

PLoS One. 2019 Jun 7;14(6):e0217967. doi: 10.1371/journal.pone.0217967. eCollection 2019.

Abstract

Kernel regression models have been used as non-parametric methods for fitting experimental data. However, due to their non-parametric nature, they belong to the so-called "black box" models, indicating that the relation between the input variables and the output, depending on the kernel selection, is unknown. In this paper we propose a new methodology to retrieve the relation between each input regressor variable and the output in a least squares support vector machine (LS-SVM) regression model. The method is based on oblique subspace projectors (ObSP), which allows to decouple the influence of input regressors on the output by including the undesired variables in the null space of the projection matrix. Such functional relations are represented by the nonlinear transformation of the input regressors, and their subspaces are estimated using appropriate kernel evaluations. We exploit the properties of ObSP in order to decompose the output of the obtained regression model as a sum of the partial nonlinear contributions and interaction effects of the input variables, we called this methodology Nonlinear ObSP (NObSP). We compare the performance of the proposed algorithm with the component selection and smooth operator (COSSO) for smoothing spline ANOVA models. We use as benchmark 2 toy examples and a real life regression model using the concrete strength dataset from the UCI machine learning repository. We showed that NObSP is able to outperform COSSO, producing stable estimations of the functional relations between the input regressors and the output, without the use of prior-knowledge. This methodology can be used in order to understand the functional relations between the inputs and the output in a regression model, retrieving the physical interpretation of the regression models.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Algorithms
  • Artificial Intelligence
  • Least-Squares Analysis*
  • Machine Learning*
  • Models, Statistical*
  • Support Vector Machine*

Grants and funding

This research is supported by the following fund: Bijzonder Onderzoeksfonds KU Leuven (BOF): SPARKLE - Sensor-based Platform for the Accurate and Remote monitoring of Kinematics Linked to E-health #: IDO-13-0358; The effect of perinatal stress on the later outcome in preterm babies #: C24/15/036; TARGID - Development of a novel diagnostic medical device to assess gastric motility #: C32-16-00364. Fonds voor Wetenschappelijk Onderzoek-Vlaanderen (FWO): Hercules Foundation (AKUL 043) ‘Flanders BCI Lab - High-End, Modular EEG Equipment for Brain Computer Interfacing’. Agentschap Innoveren en Ondernemen (VLAIO): 150466: OSA+. Agentschap voor Innovatie door Wetenschap en Technologie (IWT): O&O HBC 2016 0184 eWatch. imec funds 2017. imec ICON projects: ICON HBC.2016.0167, ‘SeizeIT’. Belgian Foreign Affairs-Development Cooperation: VLIR UOS programs (2013-2019). EU: European Union’s Seventh Framework Programme (FP7/2007-2013). The HIP Trial: #260777. ERASMUS +: INGDIVS 2016-1-SE01-KA203-022114. European Research Council: The research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013) / ERC Advanced Grant: BIOTENSORS (n_ 339804). This paper reflects only the authors’ views and the Union is not liable for any use that may be made of the contained information. EU H2020-FETOPEN ‘AMPHORA’ #766456. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.