# Linear regression with unusual error terms

Suppose that $$Y = a + b K + X$$ with $$0 < X$$ and $$0 < K$$ where $$X, Y$$ and $$K$$ are random variables. What are then the expectations of the intercept and slope in the case of a linear regression of $$Y$$ on $$K$$?

• it depends on the correlation between $X$ and $K$. Sep 20, 2019 at 13:42
• You need to correct your question to indicate that you are interested in the expectations of the estimated intercept and slope. As it stands, the question is ambiguous. Sep 20, 2019 at 14:38
• In conducting linear regression one generally is concerned about the conditional responses and therefore the expectations conditional on $K.$ Those can be computed quite generally, but the full expectations--accounting for the distribution of $K$--cannot be generally computed: there's no nice formula for them. Which are you asking for?
– whuber
Sep 20, 2019 at 19:45
• To AdamO X and K are independent. Sep 21, 2019 at 8:37
• To Isabella Ghement I am interested in the expectations of the estimated intercept and slope. Sep 21, 2019 at 8:39

Just to be sure we share a common understanding of the question, I suppose you mean you have independent data $$(X_i, K_i, y_i)$$ where $$(K_i,X_i)$$ are $$n$$ independent and identically distributed random variables (you indicate they are positive, but this does not matter), the individual $$(K_i, X_i)$$ are independent (as you state in a comment), and you use ordinary least squares to estimate $$a$$ and $$b$$ in the model

$$y_i = a + b K_i + X_i.$$

Now, because you assumed $$X \gt 0,$$ necessarily $$E[X_i] \gt 0$$ (assuming these expectations are finite). This is where your question differs from the usual regression setting. The $$X_i$$ play the role of the "errors" in the model, but because they do not have a zero expectation, their expectation must be added to the constant $$a.$$

One way to see this is to write $$\varepsilon_i = X_i - E[X_i] = X_i - \mu$$ (where $$\mu=E[X_i]$$ is the common expectation). These random variables are now legitimate errors because they have zero expectations. Writing for the common expectation, your model is the same as

$$y_i = a + b K_i + (\mu + \varepsilon_i) = (a + \mu) + b K_i + \varepsilon_i.$$

Thus, as always in least squares regression,

the expectation of the slope estimate is $$b$$ and the expectation of the intercept estimate is the true intercept, here seen equal to $$a + \mu.$$

Unless you know $$\mu,$$ you cannot determine $$a$$ from this model: you can only estimate the sum $$a+\mu.$$ (It sounds like your simulations used zero-mean variables $$X_i,$$ contrary to the assumptions stated in the question. Indeed, the expectation of the estimate of $$a$$ equals $$a$$ if and only if $$\mu=0.$$)