Radiology ExamWeb: development and implementation of a national web-based examination system for medical students in radiology

Rationale and objectives: Faculty are often limited in time, knowledge, and resources to develop efficient, effective, and valid computer-based examinations to evaluate students. Our purpose was to develop a web-based pool of standardized National Board of Medical Examiners (NBME)-format, peer-reviewed, and peer-tested questions based on the Alliance of Medical Student Educators in Radiology (AMSER) National Medical Student Curriculum to evaluate the radiologic knowledge of medical students.

Materials and methods: Members of the AMSER Electronics Committee submitted questions they had written for their institutions and later developed a 113-question standardized examination. Questions were edited by 24 subspeciality editors and then further edited by the authors to NBME recommendations. Software was developed using commercially available software (www.ExamWeb.com) with extensive modifications and additions following initial deployment. Students take examinations online and receive their scores immediately. Items were validated by identifying those answered >30 times and analyzing the following: number of times deployed, number of times correctly answered, distractor-specific breakdown, difficulty level (P), and point biserial coefficient (rbi).

Results: Radiology ExamWeb (REW) is available online with 3500 registered students from 65 institutions and 1800 active questions. Instructors can create examinations or use "shared examinations" made by another instructor but enabled for other institutions to administer or modify. More than 300 shared examinations have been developed. The AMSER curriculum was converted into database format and crosschecked with question items to ensure that the question pool adequately covered the spectrum of the curriculum. An AMSER standardized examination has been developed and deployed within REW.

Conclusions: REW has provided medical student educators with the means to evaluate students in a systematic way, using a nationally edited and regularly reviewed web-based process.