Background and purpose: Efficient abstract scoring for congress presentation is important. Given the emergence of new study methodologies, a scoring system that accommodates all study designs is warranted. We aimed to assess the equivalence of a simplified, 2-question abstract grading system with a more complex currently used system in assessing abstracts submitted for orthopedic scientific meetings in a serial randomized study.
Methods: Dutch Orthopedic Association Scientific Committee (DOASC) members were randomized to grade abstracts using either the current grading system, which includes up to 7 scoring categories, or the new grading system, which consists of only 2 questions. Pearson correlation coefficient and mean abstract score with 95% confidence intervals (CI) were calculated.
Results: Analysis included the scoring of 195 abstracts by 12-14 DOASC members. The average score for an abstract using the current system was 60 points (CI 58-62), compared with 63 points (CI 62-64) using the new system. By using the new system, abstracts were scored higher by 3.3 points (CI 1.7-5.0). Pearson correlation was poor with coefficient 0.38 (P < 0.001).
Conclusion: The simplified abstract grading system exhibited a poor correlation with the current scoring system, while the new system offers a more inclusive evaluation of varying study designs and is preferred by almost all DOASC members.