Expected Agreement Coefficient for Norm-Referenced Tests With Classical Test Theory

Abstract

There are two types of agreement coefficients for psychological test scores: norm-referenced 
and criterion-referenced agreement coefficients. These coefficients were derived within the 
framework of generalizability theory that is known for its theoretical and practical complexities. Under the framework of classical test theory, the paper derived the norm-referenced agreement coefficient. This derivation was based on the assumption of randomly equivalent test 
forms. The resulted expected agreement coefficient was different from its counterpart in generalizability theory. However, the estimators of this norm-referenced agreement coefficient were 
equal under the two frameworks to coefficient alpha reliability