Everyday life is filled with decision-making, from the clothes we put on in the morning to what we plan to cook for dinner. For a majority of individuals, the outcomes associated with these types of day-to-day decision-making typically do not result in severe consequences. Now consider the type of decisions faced by a psychologist regarding their professional responsibilities of beneficence to the community and individuals they serve. For conceptual purposes, let us discuss a case presentation. A 7-year-old child is brought into a psychologists’ office that presents with the following symptoms: frequent acts of stealing and lying; destruction of property; harm towards others; and ...view middle of the document...
Not to mention the stigmatizing damage experienced by the child and family due to the inaccurate diagnosis. Subsequently, an inaccurate diagnosis will lead to inappropriate treatment that could potentially exacerbate problems for the child and family.
As briefly demonstrated in the example above, clinical decision-making guided by assessments, is an intensive and comprehensive process. The outcome of a diagnosis has the potential to positively and/or negatively change lives. With that said, over the last several years researchers have begun to investigate the diagnostic characteristics (i.e. accuracy) of an assessment through various statistical procedures. One way to evaluate and improve upon the performance of an assessment is by using the Receiver Operating Characteristic (ROC) analysis. The ROC analysis is considered a procedure used to estimate the diagnostic properties of an assessment, beyond the standard reliability and validity coefficients (DeLong, DeLong, & Clarke-Pearson, 1988; Metz, 2006). According to Szmukeler, Everitt, and Leese (2012), ROC curve analysis provides a means for comparing how well a screener or diagnostic assessment classifies individuals into dichotomous categories (i.e. depression vs. no depression). The benefit of employing a ROC analysis is that it considers the criterion required to make a clinical diagnosis, while simultaneously estimating an assessments’ ability (or in some cases, the inability) to detect the presence of a disorder (Fawcett, 2006).
Grounded in signal detection theory, ROC analysis has given researchers a simple, yet efficient means for evaluating diagnostic characteristics of an assessment such as true and false positive rates, as well as false positive and negative rates (Gohen, 2007; Youngstrom, 2014). Despite the simplistic concepts embedded within ROC analysis, one must fully understand the integrated process before correctly using it in research or practice. Therefore, the purpose of this paper is to first, provide a summary of historical researchers and their work that contributed to the development of ROC analysis. Second, this paper will discuss some of the basic underlying components of ROC analysis. As ROC analysis has been used across a variety of disciplines, the content of this paper will focus primarily on assessments that identify emotional and/or behavioral disorders in youth. Last, a practical application of ROC analysis will be explored.
Historical Contributions to the Development of ROC Analysis
Gustav Theodor Fechner
Several historical studies have contributed to the conceptualization of ROC analysis, with research dating back to the mid 1800’s. Gustav Theodor Fechner, founder of psychophysics, was one of the first researchers to identify problems associated with response discrimination (Swets, 1973). Fechner introduced concepts know as the difference threshold and the absolute threshold (as cited in Swets, 1973). The difference threshold is the...