"Evidence-based checklists" for identifying predatory journals have not been assessed for reliability or validity: An analysis and proposal for moving forward

Background: Predatory journals (PJs) pose a threat to the quality/integrity of scientific publishing. Checklists have been proposed to identify PJs, but few are "evidence-based". This study's objective was to search for/assess evidence-based checklists (EBCs) for reliability and validity, based on a new consensus definition of PJs.

Methods: A published methods guideline for scale development was used to identify how many steps were completed in the generation of identified EBCs. Items from each EBC were compared against the consensus definition, and a list of items to be considered in the creation of a composite EBC to identify PJs was generated.

Results: Four EBCs were identified. None of these had completed the first of the nine steps for scale development and validation. Forty-seven items from the four EBCs were assessed against the consensus definition, of which 28 items fell within the definition. A proposed composite EBC was created from items matching components of the consensus definition.

Conclusion: EBCs to detect PJs lack assessment of reliability and validity. To a varying degree, the EBCs contain items that match the scope of the new consensus definition of PJs. With the recent consensus definition, EBCs creators now have a clearer target, and can make adjustments.