Intro Here comes the intimidating part. Most people won't need to delve into this section except to understand that Discrepancy Calculator's method draws upon accepted best practices. Others of you might wish to understand the mathematics or the computer programming in greater detail. Discrepancy Calculator's method of reporting severe discrepancy draws upon conclusions from the US Department of Education - Special Education Programs (USDE -- SEP) workgroup convened on learning disabilities assessment in 1983. The recommendation of that group was a regression analysis formula and cutoff line below which one could be 95% confident a student's achievement test score is significantly different from the expected achievement predicted upon his or her IQ. The USDE-SEP formula is reproduced below. This is referred to as the 'Reynolds Formula' in Discrepancy Calculator.  where Zyc = Cutoff score Zx = Cognitive test score in Z-units Rxy = Correlation between the two tests Rxx = Reliability coefficient for the cognitive test Ryy = Reliability coefficient for the Achievement test In this formula, (Zx*Rxy) is the regressed IQ score, the product of actual IQ and the test-to-test correlation. Subtracting from the regressed IQ... (1.96 * Sqrt ( 1 - Rxy ^ 2 ) ) determines the standard error estimate of the regressed IQ at a 95% certainty level. The rest of the equation completes the creation of a critical threshold score by subtracting the standard error of the gap in scores to account for test unreliability. The end result is Zyc, which is a cutoff score based on the individual's IQ that points to a statistically significant achievement level (p<=.05) below which there is a severe discrepancy. Modified Reynolds This alternative was developed by the Utah Office of Education and Utah State University. Employing a one-tailed test of significance is said to be a more appropriate hypothesis test in this application Test-to-Test Correlation An estimate of correlation between two tests is the square root of the reliability. The question becomes how do we combine the separate reliabilities of two tests so that we can take the square root of it? Several methods were offered by authors of articles I found as I researched Discrepancy Calculator's algorithm. The assumption is the two tests are parallel. That is they are measuring the same thing. If this is true, then two very reliable tests will be highly correlated. As Reliability falls, more chance is injected into the answers so the correlation between the tests falls off as well. The estimation methods work in different ways so they produce different values. It is hard to say if one is better than another and under what conditions that might be true. Unable to choose just one method for the app, i decided to make five available and let the user decide which is the best one. Sqrt ( R1 * R2 / 2) The square root of one half the product of the two reliability coefficients. This method seems to produce values consistent with correlations reported by test publishers. Sqrt ( ( R1 + R2 ) / 2 ) This is the square root of the mean reliability coefficient. Produces a value that is higher than either of the two reliability coefficients ( R1 + R2 ) / ( 1 + R1 + R2 ) This is a variation of what is known as the Spearman-Brown Correction (Revelle, p. 215). It seems to produce values consistent with correlations reported by test publishers. .7071 * Sqrt (R1 * R2 ) From Dumont & Willis Sqrt ( ( R1 + R2 ) / ( 1 + R1 + R2 )) Variation on the Spearman-Brown variation which produces a greater value than Spearman-Brown but less than the square root of the mean (#2) The effect of differing correlations is that as the correlation increases in value, the program is more likely to report a severe discrepancy. How to Use Discrepancy Calculator: go to the user guide . Need to Buy the App? open a new page showing the Discrepancy Calculator Listing on the Google Play Store. References Baer, Richard D. (undated internet). Issues in severe discrepancy measurement: A technical assistance paper for special educators. link to pdf document. Baer, Richard D. (undated internet). Utah Estimator. Link to web page. Brown, Stan (2012). Math course pages by Stan Brown. Tompkins Cortland Community College. Inference: one-tailed or two-tailed hypothesis test? Link to web page. Dumont, Ron & Willis, John O. (undated internet). Severe discrepancy determination by formula. Link to web page. Oosterhof, Albert (1999). Correlation: Application to reliability. Link to power point presentation. Revelle, William (undated internet). Introduction to psychometric theory with applications in R. Chapter 7., Reliability. Link to web page. Wright, Jim (2002 Internet). Best practices in calculating severe discrepancies between expected and actual academic scores: a step-by-step tutorial. Link to web page. |
Home > Discrepancy Calculator >