•  
  •  
 

In God We Trust, All Others Bring Data: A Bayesian Approach to Standard Setting

Abstract

Standard setting is an inherent part of pass/fail decisions in assessment. Although various standard setting methods are available, they all have their limitations and no method provides a golden solution to all our standard setting headaches. Some methods require potentially labor-intensive standard setting panels of judges who have specific knowledge. Other methods require student cohorts of ‘sufficient’ size. However, small cohorts are quite prevalent in medical programs across the globe, and standard setting panels are not always feasible due to logistic or financial constraints or may result in inadequate judgments due to bias or a lack of specific knowledge. This manuscript presents a new standard setting method, which is based on the Bayesian principle of updating our knowledge or beliefs about a phenomenon of interest with incoming data, uses information that is not considered in methods already available and can be applied to both small and larger cohorts regardless of whether standard setting panels are available. As demonstrated in this manuscript through a worked example, the new method is easy to implement and requires only a minimum of calculations which can be done in zero-cost, user-friendly Open Source software. Options for future research comparing different standard setting methods are discussed. © 2020 King Saud bin Abdulaziz University for Health Sciences

This document is currently not available here.

Share

COinS