5
$\begingroup$

I am interested in an inferential test available in R which tests whether Pearson's $r = 1$ instead of whether $r = 0$. It would be good if the test allowed the correlation matrix and number of participants as input variables.

$\endgroup$
  • $\begingroup$ Can you clarify your situation? Why are you wondering if the correlation is 1? As @eric_kernfeld notes, this may not be what you really need. $\endgroup$ – gung - Reinstate Monica Dec 2 at 21:57
  • $\begingroup$ If the correlation disattenuated for the reliability of each variable equals 1 this would mean that the correlated processes have a common mechanism which operates behind them. My hypothesis was that processes are somewhat specific relative to each other. So H0 : r = 1 and H1: r < 1. This is why I needed to test whether H0 is true. $\endgroup$ – User33268 Dec 2 at 23:46
  • 3
    $\begingroup$ It sounds like you have in mind a model of the form $(x_i,y_i)=(u_i, \alpha+\beta u_i)+(\delta_i,\epsilon_i)$ where $u_i$ is a random variable and $(\delta_i,\epsilon_i)$ is an error ("disattenuation"?) independent of $u_i$ and you want to compare that to some alternative model involving a more complicated relationship. $\endgroup$ – whuber Dec 3 at 0:16
  • $\begingroup$ In which case (following @whuber), testing for $(\alpha, \beta) = (0,1)$ (given scaled $x,y$) might reduce to testing correlation? $\endgroup$ – Nutle Dec 3 at 12:08
  • 1
    $\begingroup$ Are you really sure you want an *H0 of rho = 1? This would seem to imply both that the data are perfectly correlated and that there is no residual error in the values. Perhaps you want a linear regression, and test if beta is 1? Or maybe you just want to test if rho is 0? Maybe you can give an example data set for which you would want to reject H0 and one for which you would not want to reject H0. $\endgroup$ – Sal Mangiafico Dec 3 at 17:48
23
$\begingroup$

I would argue that there is not any testing to do. If the sample correlation is not 1, then you reject $H_0: \rho=1$ with certainty.

Having a correlation of 1 means that the points cannot deviate from a diagonal line the way that they can when $\vert \rho \vert < 1$.

EDIT

set.seed(2019)
x <- rexp(1000)
y <- 3*x
plot(x,y)
V <- rep(NA,10000)
for (i in 1:length(V)){

  print(i)
  idx <- sample(seq(1,length(x),1),replace=T)
  V[i] <- cor(x[idx],y[idx])
}
summary(V)

With the points of the scatterplot locked to the diagonal line $y=3x$, every single sample correlation is 1. You can try this out with other distributions and sample sizes.

Where this gets interesting---and I'm not completely sure of the math at the population level---is when I set a Gaussian copula to have a parameter of 1.

library(copula)
set.seed(2019)
gc <-ellipCopula("normal", param = 1, dim = 2)#, dispstr = "un")
norm_exp <- mvdc(gc,c("norm","exp"),list(list(mean=0,sd=1),list(rate=1))) 
V <- rep(NA,10000)
for (i in 1:length(V)){
  print(i)
  D_ne <- rMvdc(1000, norm_exp) 
  x <- D_ne[,1]
  y <- D_ne[,2]
  V[i] <- cor(x[idx],y[idx])
}
plot(x,y)
summary(V)

I still don't think this relationship gives a population Pearson correlation of 1 (the relationship is perfectly monotonic but not linear), but this result surprised me. I expected another plot of a straight line.

To defend my assertion that the population Pearson correlation is not 1, I refer to theorem 4.5.7 on pg. 172 of the second edition of Casella & Berger's Statistial Inference: "$\vert \rho_{XY}\vert=1$ if and only if there exist numbers $a\ne0$ and $b$ such that $P(Y = aX+b)=1$." Since the relationship between my $X$ (the normal variable) and $Y$ (exponential) is nonlinear, there can be no such $a$ and $b$.

Casella, George, and Roger L. Berger. Statistical Inference. 2nd ed., Cengage Learning & Wadsworth, 2002.

$\endgroup$
  • 12
    $\begingroup$ This is such a simple (and correct) answer that it makes me wonder whether that's really what OP needed to ask... $\endgroup$ – eric_kernfeld Dec 2 at 21:34
  • $\begingroup$ Surely if the true "population" correlation coefficient $\rho$ was $1$, by sampling error you can still get a sample where the points are not in perfect alignment with the population assumption and hence $r < 1$ even though $\rho=1$. $\endgroup$ – Paul Hewson Dec 4 at 9:41
  • 1
    $\begingroup$ @PaulHewson The points are locked to some diagonal line. Try this out with some software. I’ll edit my answer to include my own code showing this. $\endgroup$ – Dave Dec 4 at 10:57
1
$\begingroup$

Using the Fisher Z-transform is one way of doing this (usually used for confidence intervals), bootstrapping would be another. Here's a brief article for Fisher Z transform for Pearson Product Moment Correlation Coefficient https://www.statisticshowto.datasciencecentral.com/fisher-z/

$\endgroup$
  • $\begingroup$ This should be equivalent to computing the confidence interval for the correlation coefficient and check whether it includes 1. I wonder whether for the upper limit of the interval a $2\alpha$ confidence level should be used, because this is only a one-sided problem. Side note: the function CIr in the R package psychometric implements the confidence interval computation. An easier to follow reference than the above site is ncss-wpengine.netdna-ssl.com/wp-content/themes/ncss/pdf/… $\endgroup$ – cdalitz Dec 2 at 16:28
  • 7
    $\begingroup$ It is impossible for the upper limit of confidence for the Fisher Z transform ever to overlap $1,$ because $1$ corresponds to an infinite value of $Z.$ Thus this test is just a waste of effort. No amount of bootstrapping will produce an infinite upper limit, either. $\endgroup$ – whuber Dec 2 at 17:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.