Box 1 Criticisms of the Situational Judgement Test
- By: Jason Walsh and Benjamin Harris
Ability to discriminate—The final SJT pilot used seven tests of 60 questions and was taken by 6782 final year medical students. All tests showed a narrow spread of scores, with mean scores ranging from 399 to 414 points (78%-81%) and standard deviations ranging from 17 to 20 points. Candidates’ scores tended not to stray far from the mean, and the discriminatory power of the test is low. If candidates are split into deciles, as was done in the pilot tests, a difference of 1% will span a single decile at the peak of the curve—that is, the test is statistically volatile.
Weighting—As students can score 0-50 points on the SJT, it carries far more weight on the Foundation Programme applications than academic performance, as the educational performance measure is measured on a 34-50 point scale. For example, a student in the lowest decile in his medical school cohort, with no