COMPCOR: A Computer Program for Comparing Correlations Using Confidence Intervals

N. Clayton Silver, PhD

Associate Professor, Department of Psychology, University of Nevada, Las Vegas, 4505 S. Maryland Pkwy, Las Vegas, NV 89154-5030, USA; E-mail: fdnsilvr@unlv.nevada.edu

INTRODUCTION

Comparing independent or dependent correlations is often based on standard statistical significance tests.^{1,2,3,4,5,6,7} Independent correlations come from different samples. For example, suppose that a school administrator is interested in determining if there was a difference between the correlations of the mathematics scores on the Iowa Test of Basic Skills (IT) and scores on the Children’s Memory Scale (CM) for grades 2 and 6 (H_o: ρ₁=ρ₂). If the correlation for grade 2 was .50 and the correlation for grade 6 was .20 with sample sizes of 100 and 200, respectively, then the z-test for independent correlations would equal 2.79, p<.01. The conclusion would be that there is a significantly higher correlation between the mathematics scores of the IT and CM scores for grade 2 than for grade 6 children. Although the Fisher’s z-test for examining the difference between independent correlations is shown in many standard statistics textbooks,¹ it is not usually contained in the standard statistical packages unless a researcher writes a separate program for performing it.

Dependent correlations, however, are those contained within the same sample. One hypothesis consists of testing the difference between two dependent correlations with one element in common (H_o:ρ₁₂=ρ₁₃). For example, suppose that the same administrator is interested in determining if the correlation between the mathematics scores on the IT would be significantly higher with CM scores (r=.60) than with the overall scores of the Montreal Battery of Evaluation of Musical Abilities (MBEMA) (r=.30) for 100 grade 5 children. Moreover, suppose that the correlation between the scores of the CM and MBEMA was .20. There are number of procedures for testing the null hypothesis of ρ₁₂=ρ₁₃, that either compare the correlations using the t distribution2 or via Fisher’s z’ transformation which purportedly distributes out as z.3 Research indicated that they were deficient under certain conditions with regard to Type I error rate and power.4 Consequently,5 offered Method D2 as an alternative to the standard techniques. They provided this alternative in R and S-PLUS programs. Nevertheless, using the3 z-test, the value was 2.832, p<.01 indicating that the correlation between mathematics scores on the IT and CM scores was significantly higher than the correlation between mathematics scores on the IT and MBEMA scores for grade 5 children.

A second hypothesis consists of testing the difference between two dependent correlations with no elements in common (H_o:ρ₁₂=ρ₃₄). Suppose that the administrator is now interested in determining if the correlation between the mathematics scores on the IT and CM scores would be higher (r=.50) after a brief memory skill course (e.g., mnemonics) than before one for grade 4 children (r=.30). Here is a hypothetical correlation matrix for a sample size of 50:

Using the procedure,³ the z-test value was -1.75, p>.05. This indicates that there was no statistically significant difference between the correlations of the mathematics scores on the IT and CM scores before and after the mnemonic intervention for grade 4 children. In a simulation of four possible procedures for testing the null hypothesis of ρ₁₂=ρ₃₄, which included the z-test,³ one procedure was entirely too liberal, whereas the other three were a bit conservative when the predictor-criterion correlation was low.⁶ Nevertheless, the best significance test procedures for testing dependent correlations with zero and one element in common, based upon their findings,⁶ were programmed for Windows.⁷

Although statistical significance tests are used for testing these hypotheses, more emphasis has been placed on confidence intervals for performing the same task. The confidence interval separately provides the magnitude and precision of the particular effect, whereas these characteristics are confounded in standard hypothesis testing p values⁸ provided confidence interval techniques which purportedly have better control of Type I errors and have more power than the standard statistical significance tests. Although many of these techniques have been programmed in R,⁹ and recently in SAS and SPSS as separate programs,¹⁰ the problem is that many researchers who are basic users of these packages or do not use them at all, may have difficulty in applying these programs. In some cases, researchers may resort to computing these techniques by hand. Therefore, in order to make these confidence interval approaches more generalizable to researchers, the purpose of the user-friendly, stand-alone program was to compute them for testing differences between: a) independent correlations;⁸ and b) two dependent correlations with either zero or one element in common⁸ in a Windows platform.

DESCRIPTION

The user is queried interactively for the particular test, correlations, sample size, and the confidence interval probability (e.g., 95%). The normal curve value associated with computing the confidence interval for the individual correlations was obtained using the algorithm by.¹¹ The program responds with a restatement of the input correlations, sample size, the confidence interval for the individual correlations, the confidence interval for testing the differences between correlations and a brief statement mentioning that confidence intervals containing zero are non-significant. The program is written in FORTRAN 77, using the GNU FORTRAN compiler, and runs on a Windows PC or compatible. The output is contained in COMPCOR.OUT.

Sample outputs based upon the hypothetical scenarios are given in Tables 1-3. The output indicates there are no differences in the general conclusions using the confidence interval approach⁸ and the standard statistical significance tests.⁶ Although there were no differences in the general conclusions, given the findings of⁷ in terms of Type I error rates and power,⁸ it is still important for researchers to have a potentially better option at their disposal.

AVAILABILITY

COMPCOR.FOR and the executable version (COMPCOR.EXE) may be obtained at no charge by sending an e-mail request to N. Clayton Silver, Department of Psychology, University of Nevada, Las Vegas, Las Vegas, NV 89154-5030 at fdnsilvr@unlv.nevada.edu. 

CONFLICTS OF INTEREST

The authors declare that they have no conflicts of interest.

1. Howell DC. Statistical Methods for Psychology. 8^th ed. Belmont, CA: USA Wadsworth; 2010.

2. Williams, EJ. The comparison of regression variables. J R Stat Soc Series B Stat Methodol. 1959; 396-399.

3. Dunn OA, Clark VA. Correlation coefficients measured on the same individuals. J Am Stat Assoc 1969; 64: 366-377. doi: 10.1080/01621459.1969.10500981

4. Hittner JB, May K, Silver NC. A Monte Carlo evaluation of tests for comparing dependent correlations. J Gen Psychol. 2003; 130: 149-168. doi: 10.1080/00221300309601282

5. Wilcox R, Tian T. Comparing dependent correlations. J Gen Psy. 2008; 135: 105-112. doi: 10.3200/GENP.135.1.105-112

6. Silver NC, Hittner JB, May K. Testing dependent correlations with nonoverlapping variables: A Monte Carlo simulation. J of Exp Ed. 2004; 73: 53-69. doi: 10.3200/JEXE.71.1.53-70

7. Silver NC, Hittner JB, May K. A FORTRAN 77 program for comparing dependent correlations. App Psy Meas. 2006; 30: 152-153. doi: 10.1177/0146621605277132

8. Zou GY. Toward using confidence intervals to compare correlations. Psy Meth. 2007; 12: 399-413. doi: 10.1037/1082-989X.12.4.399

9. Diedenhofen B, Musch J. cocor: A Comprehensive Solution for the Statistical Comparison of Correlations. PLoS One. 2015 Apr 2;10(3): e0121945. doi: 10.1371/journal.pone.0121945

10. Weaver B, Wuensch KL. SPSS and SAS programs for comparing Pearson correlations and OLS regression coefficients. Beh Res Meth. 2013; 45: 880-895. doi: 10.3758/s13428-012-0289-7

11. Wichura MJ. Algorithm AS241: The percentage points of the normal distribution. App Stat. 1988; 37: 477-484. doi: 10.2307/2347330