# correlation coefficient in linear regression

My interest is to develop a relation of the correlation coefficient when the data (both the dependent and independent variables) have measurement errors.

## Intro

The measured values are related to the true / actual values by: \begin{align} \newcommand{\Var}{{\rm Var}}\newcommand{\cov}{{\rm Cov}} x_i &= x_{t,i} + \varepsilon_{x,i} \\ y_i &= y_{t,i} + \varepsilon_{y,i} \end{align} where $\varepsilon_{x,i}$ and $\varepsilon_{y,i}$ are the random measurement errors on $x_i$ and $y_i$, respectively. $x_t$ and $y_t$ are the true values, $\varepsilon_{x,i} \sim N(0, \sigma_x^2)$ and $\varepsilon_{y,i} \sim N(0, \sigma_y^2)$. Finally, $\sigma_x$ and $\sigma_y$ are known.

When the data are measured without error, the correlation coefficient is:

$$r = \frac{\cov(x_t,y_t)}{\sqrt{\Var(x_t)\Var(y_t)}}$$

## Case 1: Measurement errors are the same for each data point

I was able to derive the formula for the correlation coefficient in case of measurement errors. In this case $\sigma_x^2$, $\sigma_y^2$ and $\sigma_{x,y}$ are the same for each data point. Using the properties of variance and covariance, the correlation coefficient is \begin{align} r &= \frac{\cov(x_t,y_t)}{\sqrt{\Var(x_t)\Var(y_t)}} \tag{1} \\[10pt] &= \frac{\cov(x,y) - \sigma_{x,y}}{\sqrt{(\Var(x) - \sigma_x^2) (\Var(y) - \sigma_y^2)} } \end{align}

where $\sigma_{x,y}$ is the covariance.

## Case 2: Measurement errors are NOT the same for each data point

1. assuming $\sigma_{x,y} = 0$, then the numerator in equation $(1)$ simplifies to: $\cov(x_t,y_t) = \cov(x, y)$

2. In this case $\sigma_{x,i}$ are not identical. Hence I cannot use the property of the variance which says that:

\begin{align} \Var(x) &= \Var(x_t + \varepsilon_x) \\ &= \Var(x_t) + \Var(\varepsilon_x) \\ &= \Var(x_t) + \sigma_x^2 \end{align} So how can I derive a formula for the correlation coefficient in case the measurement errors are not identical?

• Your notation is a bit confusing to me. Are $x$ and $y$ vectors? Are the true values, e.g. $x_{t,i}$, fixed or is there some error associated with them? – Gumeo Oct 12 '15 at 9:50
• $x$ and $y$ are vectors. They are the given data. $x_{t,i}$ and $y_{t,i}$ are unknown (and it does not interest me to estimate them). You might want to check section 3 for more details: iopscience.iop.org/article/10.1086/519947/pdf --- OR let me know if I should give more details. @GuðmundurEinarsson – aloha Oct 12 '15 at 9:55
• You can put a bounty on your own question if you have the reputation, but you don't. Otherwise, it is for others to decide whether a bounty is deserved, which is almost always after some time when it appears that a good question has been unjustly neglected. I have to advise that asking for a bounty is not at all considered good practice; nor will emphasis on how much you need an answer make any difference to how able and willing people are to answer. Indeed, such comments are likely to be counter-productive. Hence I have edited out all such comments as definitely not in your best interests. – Nick Cox Oct 12 '15 at 11:35
• (A bounty can also be awarded if someone thinks that a very good answer deserves more credit, which is not the issue here.) – Nick Cox Oct 12 '15 at 11:42
• This question needs more clarification. The way that I understand this (errors are the same versus errors are not the same) is that we can describe the measurement errors in case I as being sampled from the same distribution and the measurement errors in case II as being sampled from a different distribution (a process which effectively can be describe as sampling from a single distribution en.wikipedia.org/wiki/Compound_probability_distribution). In that case there is effectively no difference between case I and case II. – Martijn Weterings Jun 29 '18 at 11:45

## 1 Answer

My understanding of your question is that you want to compute an estimate of $$V=\frac{1}{n}\sum_i(X_i+\epsilon_i-\bar{X}-\bar{\epsilon})^2$$ where $\bar{u}$ is the empirical mean of any vector $u=(u_1,\dots,u_n)$, $X_i$ are iid with variance $var(X)$, and $\epsilon_i/\sigma_i$ are iid with variance 1 and mean zero.

My answer $V$ splits into 3 different temrs: $V=V_1+V_2+V_3$, with $$V_1=\frac{1}{n}\sum_i(X_i-\bar{X})^2$$ $$V_2=\frac{1}{n}\sum_i(\epsilon_i-\bar{\epsilon})^2$$ $$V_3=\frac{2}{n}\sum_i(X_i-\bar{X})(\epsilon_i-\bar{\epsilon})$$

From your assumption $V_3$ tends to be null and $V_1$ tends to equal $var(X)$ (with my notation, my $X$ is your "$x_t$"), hence what you want is the limit of $V_2$. You observe that in the case when $\sigma_1=\sigma_2=\dots=\sigma_n=\sigma$ then $V_2$ tends to $\sigma^2$.

You really have to understand your question is about the limit (which wasn't really clear from you formulation because you mix theoretical and empirical quantities) for $V_2$. Under suitable conditions that might involve further moment convergence, I'm pretty sure $V_2$ converges to it expectation which means:

$$V_2 \rightarrow \lim_n\frac{1}{n}\sum_i\sigma_i^2$$