This is more of a question about my understanding of the concept; suppose we have a normal linear model $Y = X\boldsymbol{\beta}+\epsilon$, where $\epsilon \sim N(0,\sigma^{2}I_{n})$ and $\boldsymbol{\beta}$ is the vector of unknown parameters.

When we estimate the unknown parameter $\boldsymbol{\beta}$ using the maximum likelihood approach, are we basically finding the $\boldsymbol{\hat{\beta}}$ that maximises the probability of obtaining a set of realisations ($y_{1},...,y_ {n}$) given our covariates? E.g. if we postulate that two variables have such a relationship and we know the realisations of the response and also the covariates, we want to maximise the probability that the model spurts out the values of the realisations?

  • $\begingroup$ @Tim Thank you -- but is the 'probability of obtaining the realisations given the covariates' not interchangeable with likelihood? $\endgroup$ Commented May 5, 2017 at 20:08
  • $\begingroup$ Check stats.stackexchange.com/questions/2641/… as it answers this question. $\endgroup$
    – Tim
    Commented May 5, 2017 at 20:36