Evaluating the consistency of judgments derived through both in silico and expert application of the Cramer classification scheme

The Cramer classification scheme has emerged as one of the most extensively-adopted predictive toxicology tools, owing in part to its employment for chemical categorisation within threshold of toxicological concern evaluation. The characteristics of several of its rules have contributed to inconsistencies with respect to degree of hazard attributed to common (particularly food-relevant) substances. This investigation examines these discrepancies, and their origins, raising awareness of such issues amongst users seeking to apply and/or adapt the rule-set. A dataset of over 3000 compounds was assembled, each with Cramer class assignments issued by up to four groups of industry and academic experts. These were complemented by corresponding outputs from in silico implementations of the scheme present within Toxtree and OECD QSAR Toolbox software, including a working of a "Revised Cramer Decision Tree". Consistency between judgments was assessed, revealing that although the extent of inter-expert agreement was very high (≥97%), general concordance between expert and in silico calls was more modest (∼70%). In particular, 22 chemical groupings were identified to serve as prominent sources of disagreement, the origins of which could be attributed either to differences in subjective interpretation, to software coding anomalies, or to reforms introduced by authors of the revised rules.