2
$\begingroup$

I'm currently studying linear regression on this book "F.M. Dekking - A Modern Introduction to Probability and Statistics: Understanding Why and How" where the signal+noise model is presented:

$Y_i = \alpha\ + \beta x_i\ + U_i$

where $x_i$ are non-random, $U_i$ are independent r.v.s with $E[U_i] = 0$ and $Var(U_i) = \sigma^2$ and $Y_i$ are, therefore, independent r.v.s with $E[Y_i] = \alpha\ + \beta x_i$ and $Var(Y_i) = \sigma^2$

He writes that "$\{Y_1, ..., Y_n \}$ do not form a random sample because given that each $Y_i$ have different expectation, they have different distributions" and he writes also that "if we assume that $U_i$ are normally distributed we can use MLE to estimate $\alpha,\beta$".

Now, given this two facts, how could we use MLE on something that is not a sample? Maybe the correct interpretation is that $\{Y_1, ..., Y_n \}$ is i.n.i.d. sample, but, if it is the case, would it be ok to use MLE?

$\endgroup$
  • $\begingroup$ Check stats.stackexchange.com/questions/246047/… $\endgroup$ – Tim Oct 13 '17 at 12:54
  • $\begingroup$ Thanks for the comment Tim. I've read it all but it seems to me that actually it is mainly concerned with the random vs. fixed nature of the $X_i$ variables - anyway, the best answer is really "not for everybody" as someone wrote. What I don't understand here is the "$Y_i$ do not form a random sample" thing... did you intend that in that link there are some hints (that I therefore do not see) to cope with this? $\endgroup$ – Luca Oct 13 '17 at 13:34
2
$\begingroup$

Check the related question asking about logistic regression. The quotes you posted sound confusing, but what they are saying is that $Y_1,Y_2,\dots,Y_n$ are independent, but not identically distributed, since they have different means

$$ Y_i \sim \mathcal{N}(\mu_i, \sigma^2) $$

where $\mu_i$ is defined in terms of functional relationship $\mu_i = \alpha + \beta X_i$, what is equivalent of saying that

$$ \begin{align} \varepsilon_i &\sim \mathcal{N}(0, \sigma^2) \\ Y_i &= \alpha + \beta X_i + \varepsilon_i \end{align} $$

so $Y_1,Y_2,\dots,Y_n$ are random samples, but they are not i.i.d. Since they are random variables, we can define a likelihood function for the model describing their joint distribution

$$ L(\alpha,\beta,\sigma^2 \mid Y) = \prod_{i=1}^n \mathcal{N}(y_i; \alpha + \beta x_i, \; \sigma^2) $$

so we can estimate the parameters based on it (using MLE, or Bayesian estimation). You can also check the answers and comments in this thread for extended discussion and this thread for more formal discussion of somehow similar question.

$\endgroup$
  • $\begingroup$ Ok, so the sample is i.n.i.d. and I can use MLE to estimate the parameters $\endgroup$ – Luca Oct 15 '17 at 10:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.