Correlation coefficient between a stochastic and non-stochastic variable or, what is the difference between a non-stochastic variable and a constant?

Question

I am asking in the context of simple linear regression.

If my regressor $X_i$ is non-stochastic, should $\mathbb E(X_i) = \overline X$ or $X_i$ i.e., treat it like a constant? I presume the latter because some of the derivations, the unbiased one for example treat it as a constant by taking $X$ terms out of the expectation.

Further, what is $\mathrm{Corr}(Y_i,X_i)$ where $Y_i$ is the regressand. If we treat $X_i$ as a constant like we did previously, then the correlation should be undefined since $\mathrm{Cov}(Y_i,X_i)=0$ and ${\sigma_x}^2 = 0 \quad \because X_i - \mathbb E(X_i) = 0$. Intuitively, though I feel like it should be a number close to $\pm 1$ because they share a linear relationship.

If we go by undefined, then my question is what is correlation coeffecient? So far, I thought of it as a measure of how closely can we fit a straight line in a scatter plot but that definition seems to break down when we consider a non-stochastic variable.

What's more puzzling is that the book I am following, Basic Econometrics by Damodar Gujarati, states that the residuals $\widehat u_i$ are uncorrelated with $X_i$ because $\sum_{i=1}^n \widehat u_i \, X_i = 0$ (end of section 3.1, fifth edition).

This makes me think that correlation should not be undefined for a non-stochastic variable, thus my intuition serving right.

If we assume $X_i$ as stochastic i.e., $\mathbb E(X_i) = \overline X$, then $\mathrm{Cov}(Y_i,X_i) = \mathbb E(u_i \, X_i)$ which makes a lot more sense, but then some of the proofs that require it to be treated like a constant break down.

While writing this question, I realised that it may be that for unscripted terms i.e., constants, the expectation is itself but for scripted terms i.e., non-stochastic variable, there should be a bar, but that still doesn't solve the problems. With this interpretation, $\mathbb E(u_i \, X_i) = \overline X \,E(u_i) = 0$. Which again, doesn't make sense why two variables that share a linear relation should have zero covariance, and this still poses the problem of treating it like a constant in derivations. $\mathbb E(X_i \, Y_i)$ should $= \overline X \, \mathbb E(Y_i)$ not $X_i \, \mathbb E(Y_i)$.

Lastly, I know $\mathrm{Corr}(Y_i,X_i) \neq 0$ generally, since coeffecient of determination has a similar formula and that obviously won't always be 0.

Answer 1

To be honest, I am not really sure that I have understood all of your questions correctly (and I do not have the book you are referring to), but I will try to answer it.

The difference between nonstochastic and stochastic regressor is that you "fix" the nonstochastic one so that when you measure it multiple times, you will still get the same value. Number of days in a month is a good example: it is not constant, but it is "fixed".

This means that if $X_i$ is nonstochastic, then $\mathbb{E}(X_i) = X_i$. As for the $Corr(Y_i, X_i)$, you are correct, it is not defined. We basically say: "we have no information about the relationship", and it can be anything from $-1$ to $1$.

As for your question about stochastic regressors, $\mathbb{E}(X_i)$ can be very well approximated by $\bar{X}$ (by LLN in case of i.i.d. $X_i$), but it does not have to be equal to it. Moreover, it will most definitely be two different numbers, even with two different names: the expectation and the sample mean.

Covariance between $Y_i$ and $X_i$ can then be calculated as $$Cov(Y_i, X_i) = Cov(\beta_0 + \beta_1 X_i + u_i, X_i) \\ = Cov(\beta_0, X_i) + \beta_1 Cov( X_i, X_i) + Cov(u_i, X_i)\\ = \beta_1 Var(X_i) $$ because by the classical OLS assumptions $Cov(u_i, X_i) = 0$. I guess by plugging these values in the proofs that you were talking about would not break them down.