Study of internal medicine residents quantifies variables associated with IM-CE performance
Philadelphia, April 9, 2020 – A new study published in Academic Medicine finds that the American College of Physicians (ACP) Internal Medicine In-Training Examination® (IM-ITE®) and the United States Medical Licensing Examination® (USMLE®) Steps 1, 2 Clinical Knowledge (CK) and 3 predict performance on the American Board of Internal Medicine (ABIM) Internal Medicine Certifying Examination (IM-CE).
This is the first time the relationship between these commonly used measures of medical knowledge across the continuum of medical education has been studied using this multisource assessment data from a full complement of residents. The researchers believe the findings will help educators identify students and residents in need of additional support to address knowledge gaps, and to chart their progress on subsequent assessments with the goal of passing the IM-CE. Previous research studies have found that physicians who pass the IM-CE to become board certified deliver better patient care in many areas than those who do not.
Related to IM-CE performance, the IM-ITE was the strongest single predictor in both absolute magnitude and relative importance, and combined performance on USMLE Step 1, Step 2CK, and Step 3 were similarly strong predictors. ACGME milestones ratings of medical knowledge as well as age were statistically significant predictors, but contributed relatively little in magnitude to the overall model.
Medical knowledge is assessed for licensure (while students are in medical school) with the USMLE Steps 1 and 2 CK, during graduate medical education by Step 3 and by in-training examinations, and after training by board certification examinations. ABIM, ACP, and the National Board of Medical Examiners® (NBME®), which administer the IM-CE, IM-ITE®, and USMLE, respectively, conducted the study, “Correlations between the United States Medical Licensing Exams, ACP In-Training Examination, and ABIM Internal Medicine Certification Examination.”
Although studies of the USMLE and IM-ITE® have been conducted previously, the current publication represents the largest study of IM residents examining associations of multiple assessments of medical knowledge, including USMLE scores, IM-ITE® scores, and milestone ratings, with IM-CE performance.
“This study provides highly reliable quantitative evidence of associations among four of the most important assessments of medical knowledge from medical school through residency and beyond,” said Davoren Chick, MD, FACP, a co-author of the paper and ACP’s Senior Vice President for Medical Education. “Internal medicine residents and residency program directors can use information from the USMLEs and IM-ITE® to identify knowledge gaps and develop evidence-based, individualized learning plans.”
To assess the relationship between the IM-ITE®, USMLE, and IM-CE, researchers analyzed the performances of 9,767 Postgraduate Year One (PGY1), 11,424 PGY2, and 10,239 PGY3 U.S. internal medicine residents who took the IM-ITE in 2014 or 2015 and the IM-CE between 2015 and 2018. Researchers controlled for USMLE Steps 1, 2CK, and 3 scores, medical knowledge milestone ratings, age, gender, and medical school origin.
ACP administers the IM-ITE® to help residents assess knowledge gaps and program directors evaluate residency curricula. The IM-ITE® is widely used, with over 74 percent of U.S. internal medicine residents taking the examination. “The collaborative data and idea-sharing between our groups helped us jointly convey a clear understanding of the utility of the in-training exam in the context of other high stakes assessments,” said Dr. Miguel, Paniagua, MD, FACP, FAAHPM and Medical Advisor, NBME.
“This new research provides program directors with important insights demonstrating how performance on the USMLE and IM-ITE are predictors of performance on ABIM’s IM-CE,” said Furman McDonald, MD, MPH, Senior Vice President for Academic and Medical Affairs at ABIM and lead author of the study. “The key takeaway from this study is that if residents have knowledge gaps early in their training, program directors now have the information they need to predict how they may perform in the future, but more importantly, they can use these data to help the resident learn in the areas needed to exceed the predictions by addressing the knowledge gaps. Previous research tells us that doctors who perform well on ABIM’s initial certification exam, as a group, deliver better care. Anything program directors and residents can do to build their fund of knowledge coming out of training bodes well for the future of patient care.”
About the American College of Physicians
The American College of Physicians is the largest medical specialty organization in the United States with members in more than 145 countries worldwide. ACP membership includes 152,000 internal medicine physicians (internists), related subspecialists, and medical students. Internal medicine physicians are specialists who apply scientific knowledge and clinical expertise to the diagnosis, treatment, and compassionate care of adults across the spectrum from health to complex illness. Follow ACP on Twitter and Facebook.
ABIM Board Certified Doctors Make a Difference
Internists and subspecialists who earn and maintain board certification from the American Board of Internal Medicine (ABIM) differentiate themselves every day through their specialized knowledge and commitment to continual learning in service of their patients. Established as an independent nonprofit more than 80 years ago, ABIM continues to be driven by doctors who want to achieve higher standards for better care in a rapidly changing world. Visit ABIM's blog to learn more and follow ABIM on Facebook, Twitter and Instagram. ABIM is a member of the American Board of Medical Specialties.
About the National Board of Medical Examiners®
The National Board of Medical Examiners® (NBME®)is an independent, mission-driven organization that provides high-quality assessments for the health professions. We serve the health of the public through state-of-the-art assessment of health professionals, and we are committed to research and development in evaluation and measurement.
The NBME delivers valid information about health care knowledge and skills to score recipients to support important inferences and decisions. With over 100 years of expertise in assessment for medical licensing, NBME continues its pursuit of new and reliable approaches to advance the standard of care. Our global offerings encompass the spectrum of health professionals along the continuum of education, training, certification, and practice. Learn more at www.NBME.org.
About the United States Medical Licensure Examination®
United States Medical Licensure Examination® (USMLE®) is a three step examination for medical licensure in the United States and is sponsored by the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners® (NBME®).
USMLE provides licensing authorities meaningful information from assessments of physician competencies – including medical knowledge, communication and other clinical skills - that are important to the provision of safe and effective patient care. The program follows professional testing standards to provide fairness and equity to examinees, while providing important information to medical regulation and medical education.
The USMLE program engages medical educators and their institutions, licensing authority members, and practicing clinicians in the design, development and continuous improvement of these assessments.