Why calibrating LR-systems is best practice. A reaction to "The evaluation of evidence for microspectrophotometry data using functional data analysis", in FSI 305

Forensic Sci Int. 2020 Sep:314:110388. doi: 10.1016/j.forsciint.2020.110388. Epub 2020 Jun 27.

Abstract

In their paper "The evaluation of evidence for microspectrophotometry data using functional data analysis", in FSI 305, Aitken et al. present a likelihood-ratio (LR) system for their data. We show the values generated by this system cannot be interpreted as LRs: they are ill-calibrated and should be interpreted as discriminating scores. We demonstrate how to transform the scores to well-calibrated LRs using a post-hoc calibrating step. Also, we address criticisms of calibration posited by Aitken et al. We conclude by noting that ill-calibrated LR-values are misleadingly small or large. Therefore calibration should be measured and, if necessary, corrected for. The corrected LR-values (instead of the discriminating scores) can be used to update the prior odds in Bayes rule.

Keywords: Calibration; Feature-based; Forensic; Functional data analysis; Likelihood ratio; Microspectrophotometry; Score-based; Validation.