Computer-Assisted Scoring Coming to Step 2 CS

The United States Medical Licensing Examination® (USMLE®) program is committed to applying the best available technology and the most current assessment practices to ensure accuracy and validity for all its examination programs. Beginning October 2019, computer-assisted scoring technology will be used to score patient notes created by examinees for the USMLE Step 2 Clinical Skills (CS) examination.

The use of computer-assisted scoring will not change the cost of the exam, nor the timing with which examinees receive their scores. The computer-assisted scoring program is based on the scoring rubrics developed by experienced physician note raters. For cases in which computer-assisted scoring is used, notes with machine scores below the 50th percentile in performance will also be scored by physician patient note raters. The machine scores and physician scores will then be averaged to produce the examinee’s score on the patient note portion of the Step 2 CS exam.

Much consideration has gone into the implementation of computer-assisted scoring. Most importantly, the crucial role of clinical judgment in the evaluation of Step 2 CS performances will be preserved. Second, the examinee experience will not be adversely affected. The expert judgment of physicians will be used on an ongoing basis for the development of patient note scoring guidelines and in the training of the computer-based scoring system.

Click here for FAQs related to computer-assisted scoring. To read more about NBME’s work with natural language processing, see a recent article in Academic Medicine here.

Examinees should monitor the USMLE website for the latest information.