Performance on situational judgment tests is more closely associated with overall performance among surgical residents compared with traditional cognitive measures, such as the US Medical Licensing Examination Step 1 (USMLE1), according to the results of a study published in JAMA Surgery.
Aimee K. Gardner, PhD, from Baylor College of Medicine, Houston, Texas, and Brian J. Dunkin, MD, from Houston Methodist Hospital, also in Texas, investigated 3 screening tools used in industrial selection settings, including emotional intelligence, personality profiles, and situational judgment tests, to determine whether these could identify successful surgery residents.
These 3 tests were administered to 51 general surgery residents in postgraduate years 1 through 5 in a large general surgery residency program from July through August 2015. Associations between the test results and the performance of residents were explored through correlation and hierarchical regression analyses.
Continue Reading
The results showed that USMLE1 was significantly associated with overall performance of residents. However, emotional intelligence did not add to the validity of USMLE1 scores, and the inclusion of the personality factors did not add significant value to the testing. In contrast, the inclusion of situational judgment tests scores accounted for 15% more of the variance in performance than USMLE1 scores alone. Together, the USMLE1 and situational judgment tests scores accounted for 25% of the variance. Both these measures were significantly associated with overall resident performance (P =.03 and P =.005, respectively).
The researchers note that all the data come from a single specialty in a single institution, which hinders the generalizability of these findings. The evaluations of the tests themselves are also subject to biases prevalent in medical education settings. However, the authors suggest that these data offer support for further investigation of these 2 screening assessments on a larger scale across specialties and institutions.
Reference
Gardner AK, Dunkin BJ. Evaluation of validity evidence for personality, emotional intelligence, and situational judgment tests to identify successful residents [published online December 27, 2017]. JAMA Surg. doi: 10.1001/jamasurg.2017.5013
