According to a recent article published in the Annals of Internal Medicine, access to an electronic resource during a certification examination did not adversely affect test performance, nor did it change the specific skill or factor targeted by the exam.
Electronic resources are used among physicians on a daily basis, providing them with a wealth of information that is easily accessible. Some experts have voiced their concerns regarding the use of these resources during these high-stakes board certification exams.
However, it is still unknown whether the use of these resources will be able to differentiate between high and low abilities of the exam takers. The purpose of allowing access to these resources would be to reduce anxiety among test-takers and allow these physicians to verify information as they would normally do in the clinical setting.
Continue Reading
This randomized controlled trial included 825 physicians who were initially certified by the American Board of Internal Medicine by passing either the Internal Medicine Certification exam or the Internal Medicine Maintenance of Certification examination from 2012 to 2015.
Participants took the same modified version of the Internal Medicine Maintenance of Certification examination under 1 of 4 of these randomly assigned conditions: closed book, using typical or additional time, or open book with access to UpToDate, using typical or additional time.
The following outcomes were measured: item difficulty, item discrimination, average question response time, examination dimensionality, and test-taking strategies, including using item response theory to calculate question characteristics and analysis of variance to compare differences between conditions.
Results of this study found that participants taking the exam in closed-book conditions took significantly less time than in open-book conditions, with an average of 79.2 seconds [95% CI, 78.5-79.9 seconds] vs 110.3 seconds [95% CI, 109.2-111.4 seconds] per question. In addition, the average item discrimination was significantly higher for open-book conditions (0.34 [95% CI, 0.32-0.35] vs 0.39 [95% CI, 0.37-0.41]) per question. Last, the examination measured the same factor with or without the online resource, based on a strong single dimension.
Allowing physicians to use external resources during these high-stake exams may require increasing the length of the actual exam, decreasing the number of questions tested, or selectively choosing which questions should allow the use of these external resources. The authors suggest that further studies should be done to measure the effect to which the resource being used is warranted for other examinations.
Reference
Lipner RS, Brossman BG, Samonte KM, Durning SJ. Effect of access to an electronic medical resource on performance: characteristics of a certification examination [published online August 15, 2017]. Ann Intern Med. doi: 10.7326/M16-2843
