Background and study aims A reliable outcome measure is needed for bowel preparation quality during capsule endoscopy. Currently, no scales are adequately validated. Our objective was to update an existing small bowel preparation score, create a standardized training module, then determine its inter-rater and intra-rater reliability. Patients and methods Modification to produce standardized scoring of an existing small bowel preparation score was performed followed by development of a training module and validation to create the new Korea-Canada (KODA) score. Twenty readers from a range of backgrounds, including capsule endoscopists, gastroenterology fellows, residents, medical students, and nurses rated bowel cleanliness in 25 capsule videos consisting of 1,233 images, in duplicate 4 weeks apart, after completing the training module. Sequential images selected in 5-minute intervals during small bowel transit were rated on a scale between 0-3 based on the amount of visualized mucosa and the degree of obstruction. Reliability was assessed using estimates of intraclass correlation coefficients (ICCs). Results Intraclass correlation coefficients for inter-rater (ICC 0.81, 95 % CI 0.70-0.87) and intra-rater (ICC 0.92, 95 % CI 0.87-0.94) reliability were almost perfect among the 20 readers. Inter-rater reliability ranged between 0.72 (95 % CI 0.57-0.81) and 0.89 (95 % CI 0.79-0.93) for nurses and residents, respectively. Intra-rater reliability was greater than 0.90 for all groups except for nurses, which was still almost perfect (ICC 0.86, 95 % CI 0.79-0.90). Conclusions Almost perfect inter-rater and intra-rater reliability was observed for the KODA score. This simple score could be used for future clinical trials after completion of the training module.